DOI: 10.1148/radiol.2441061122
(Radiology 2007;244:165-173.)
© RSNA, 2007
CT Colonography: False-Negative Interpretations1
Taral Doshi, BS,
David Rusinak, MD,
Robert A. Halvorsen, MD, FACR,
Don C. Rockey, MD,
Kenji Suzuki, PhD, and
Abraham H. Dachman, MD
1 From the Department of Radiology, University of Chicago, Department of Radiology, MC 2026, 5841 S Maryland Ave, Chicago, IL 60637 (T.D., D.R., K.S., A.H.D.), Department of Radiology, Medical College of Virginia–Virginia Commonwealth University, Richmond, Va (R.A.H.); and Division of Digestive and Liver Disease, University of Texas Southwestern Medical Center, Dallas, Tex (D.C.R.). From the 2006 RSNA Annual Meeting. Received June 28, 2006; revision requested August 30; revision received October 6; accepted November 2; final version accepted December 15. Supported by the University of Chicago Department of Radiology Research Fund.
Address correspondence to A.H.D. (e-mail: ahdachma{at}uchicago.edu).
 |
ABSTRACT
|
|---|
Purpose: To retrospectively evaluate if false-negative interpretations at computed tomographic (CT) colonography are due to observer error.
Materials and Methods: This study was HIPAA compliant and had institutional review board approval, with waiver of informed consent. An initial unblinded review of CT colonographic image data was used to generate reconciliation reports for all false-negative polyp candidates 6.0 mm or larger. These findings were then verified by two experienced readers. After reports from the original study and reconciliation reports were reviewed, errors were classified as observer (measurement or perceptual) errors, technical errors (eg, those caused by insufficient distention, fluid), or not reconcilable. Per-polyp and per-patient sensitivity values were calculated for adenomas 6.0 mm or larger in the original data set and again by assuming elimination of technical and observer errors.
Results: Of the original data set of 228 available polyps, 147 were adenomas; for this subgroup, the per-patient sensitivity was 70% and 68% at 10.0- and 6.0-mm thresholds, respectively. When all histologic types were considered, 114 polyps were false-negative findings. Of these, 53% (60 of 114) were attributed to observer-related errors, and 26% were attributed to errors classified as technical. After detailed retrospective reconciliation of individual polyps (so as to exclude any potentially correctable observer error), the per-polyp sensitivity of CT colonography for adenomas 10.0 mm or larger increased to 93%, and the per-patient sensitivity increased to 91%. When observer and technical errors were accounted for, eight (5.4%) of 147 adenomas 6.0 mm or larger could not be detected. If all technical errors and observer errors were scored as true-positive findings, the sensitivity for adenomas 6.0 mm or larger would have been 95% on both a per-polyp and a per-patient basis.
Conclusion: The major contributor to error at CT colonography was observer perceptual error, while observer measurement error played a smaller role.
© RSNA, 2007
|
INTRODUCTION
|
|---|
Computed tomographic (CT) colonography has been proposed as an alternative to colonoscopy for imaging of the colon, including imaging performed for colorectal cancer screening. CT colonography is appealing because it does not involve the intravenous sedation, analgesia, or recovery time associated with conventional colonoscopy. A variable sensitivity for CT colonography has been reported. Results of some studies have demonstrated high sensitivity for CT colonography in polyp detection in high- and average-risk populations (1–4); however, other studies have indicated substantially lower sensitivities (5,6). One large clinical study by Rockey et al (7) had the added advantage of generating a more precise reference standard by evaluating colonoscopy after segmental unblinding, CT colonography, and air-contrast barium enema (ACBE) and by using the reconciliation between results of the three procedures to ensure that lesions were not missed. Few data are available concerning the reasons for false-negative interpretations at CT colonography, although a number of different explanations have been advanced, including technical and observer errors (8).
We hypothesized that false-negative interpretations at CT colonography are largely due to observer-related error. Our rationale was that if polyps are clearly identifiable in retrospect and are deemed prospectively diagnosable by experienced readers, then further training or computer-aided detection programs could potentially have a large impact on the sensitivity and positive predictive value of CT colonography. Thus, the purpose of our study was to retrospectively evaluate if false-negative interpretations at CT colonography are due to observer error.
|
MATERIALS AND METHODS
|
|---|
This study was compliant with the Health Insurance Portability and Accountability Act. The original trial had institutional review board approval, and all participants gave written informed consent. This retrospective review also had institutional review board approval, with waiver of additional consent. We obtained CT images and study worksheets containing information on polyp pathologic findings, size, and location for the 614 high-risk patients who participated in the original study by Rockey et al (7). Approximately 30% (186 of 614) of the patients were female, and the mean age for the entire 614-patient cohort was 57 years ± 10 (standard deviation).
The original study involved not only ACBE, but also same-day CT colonography and colonoscopy with segmental unblinding, followed by reconciliation of CT colonographic and optical colonoscopic findings. CT colonography was performed without stool tagging by using either manual room air or mechanical carbon dioxide insufflation and by using a primary two-dimensional (2D) reading method with three-dimensional (3D) problem solving with one of two software packages (Vitrea, version 3.2, Vital Images, Minneapolis, Minn [n = 467]; or Advanced Windows, version 4.0 or higher, GE Medical Systems, Milwaukee, Wis [n = 85]). Lesions were regarded as a match if a lesion found with one test was within one colonic segment of a lesion found with the other test and if the CT colonographic finding was within 50% of the size of the lesion found and measured at colonoscopy. The reference standard for accuracy was a final reconciliation of the unblinded lesions identified with all three imaging modalities (ACBE, CT colonography, and colonoscopy) (7).
False-Negative Reconciliation
Retrospective analysis was performed at a workstation (Vital Images) with software (Vitrea 2, version 3.7; Vital Images). An initial review and comparison of CT colonographic and optical colonoscopic findings was performed by unblinded observers (T.D. and D.R.) who used all original study data, including CT colonography reports, ACBE reports, colonoscopy reports, and reconciliation forms. With these resources, true-positive findings and diminutive polyps (
6.0 mm) were eliminated from the candidate pool, and reconciliation reports were generated for false-negative polyp candidates 6.0 mm or larger. Reports consisted of a screen capture of the CT images and their 3D representations.
For each false-negative polyp, two experienced readers (R.A.H., with experience of 350 proved cases, and A.H.D., with experience of 700 proved cases) independently performed an initial reading. Initial retrospective analysis utilized primary 2D analysis and 3D problem solving. At this stage, the reader was aware that a polyp 6.0 mm or larger was present and had been missed but was unaware of the polyp's exact size and location. If no polyp match was found by using 2D analysis, the reader was unblinded to the polyp segment, and the segment was reexamined with 2D and 3D analysis. The reader was asked if a polyp was present, if it was prospectively diagnosable, and to measure its size.
After the readers had recorded their findings, they were unblinded to previously generated reports and to reconciliation results from the original study. Discrepant findings were reconciled by using data from the original study and by reexamination of the specific and neighboring colonic segments and individual polyp candidates. The experienced readers made the final determination in consensus as to whether a lesion that was identified could reasonably correspond to the lesion missed in the original study and as to whether the lesion was prospectively diagnosable. In making this decision, the reader was able to view the entire CT colonographic study and to compare supine and prone images in 2D and 3D and manipulate window and level settings in a manner that would be used in routine clinical practice. The assignment of confidence to the interpretation was not quantified but was guided by the readers' normative interpretation thresholds. Difficult polyps were classified through consensus of the two experienced readers.
We (T.D., A.H.D., and R.A.H.) created specific rules to measure polyp size on the basis of the single largest dimension on either magnified transverse 2D or 3D endoluminal views, as recommended by recent guidelines (9,10). These guidelines were published subsequent to the original study and thus were not utilized in that study. For the purposes of reconciliation of lesions in this study, we continued to use the same size rule, such that a lesion that was within 50% of the size reported at the time of colonoscopy was considered the same lesion, but we used the new CT colonography measurements made at retrospective review.
If our reconciliations made on the basis of these revised criteria were still equivocal—for example, because many polyp candidates were clustered near each other in the same colon segment at CT colonography—we deferred the decision to coincide with the reconciliation result in the original study in deciding which polyp candidate matched the polyp found at colonoscopy. With this method, a match for each false-negative finding was established. Our reconciliations agreed with those of the original trial for all polyps. Because we found more polyp candidates (eg, retained stool) than the original reader, for about 20% of CT colonographic examinations we deferred to the original trial reconciliation rather than make an arbitrary decision.
Assignment of Error Type
Each false-negative polyp was classified (by T.D., D.R., A.H.D., and R.A.H.) into one of three major categories (observer error, technical error, or not reconcilable). For polyps where a match with a high degree of confidence between CT colonographic and optical colonoscopic findings was made by readers, the false-negative finding was recorded as an observer error. False-negative findings that resulted from a reader's measurement of a polyp at CT colonography that was less than 50% of the measurement at colonoscopy were recorded as observer measurement errors. Technical problems were another source of error, independent of the reader, that led to CT colonography false-negative findings. In most CT colonography examinations with technical errors, a polyp candidate was either not found or was found but was not diagnosable owing to low confidence in reconciliation. Technical errors were classified as either artifact (breathing artifact or streak artifact from hip prosthesis), insufficient distention, excessive fluid, excessive residual stool, or scan field of view errors (Fig 1). Scan field of view errors applied to false-negative polyps that were not visible at CT colonography because areas of the colon were not included on the original scan. The not-reconcilable category of false-negative findings included polyps that were not found at retrospective analysis or were deemed not prospectively diagnosable by experienced readers.
View larger version (84K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1a: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
View larger version (84K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1b: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
View larger version (67K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1c: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
View larger version (77K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1d: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
View larger version (77K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1e: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
View larger version (86K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1f: Error classification. (a) Observer error: 7-mm adenoma missed at CT colonography in original study. Left: Prone transverse soft-tissue image shows polyp (arrow). Right: 3D endoluminal view of polyp. (b) Observer measurement error: In original study, polyp (arrow) was measured as 7 mm at CT colonography and 15 mm at colonoscopy. Consensus retrospective CT colonographic measurement was 9.5 mm. Left: Supine transverse soft-tissue image. Right: Endoluminal view. (c) Artifact error: Severe respiratory motion caused by breathing during both supine sagittal (left) and prone transverse (right) acquisitions prevents polyp identification. Error classification. (d) Distention error: Arrows highlight limited distention that prevents polyp identification in supine (right) and prone (left) transverse views. (e) Excessive fluid error: 15-mm polyp found at colonoscopy was not visible in cecum at CT colonography owing to excess fluid (arrows) in colon. Left: Supine transverse view. Right: Prone transverse view. (f) Excessive stool error: 100-mm polyp in cecum is indistinguishable from large clumps of stool which appear throughout the colon. Arrow pointing up = stomach, arrow pointing right = small bowel; all other arrows = clumps of stool. Left: Supine coronal view (very small arrow in upper left corner is computer cursor). Right: Supine transverse view.
|
|
Statistical Analysis
First, we (T.D. and D.R.) determined the per-patient sensitivity for CT colonography considering only adenomas and cancers, with no manipulation of the original study's reconciliation results. This was performed to compare the original study's results with the results of our retrospective analysis when considering the polyp histologic features relevant to colorectal cancer screening.
In clinical practice, patients with true-positive findings at CT colonography would be referred for colonoscopy (9); therefore, only patients with false-negative findings but no simultaneous true-positive findings were used for our false-negative per-patient analysis. Rules were established to select the defining polyp for per-patient statistical analysis in patients with multiple false-negative polyps. First, the largest of the false-negative polyps according to the colonoscopic measurement in the original study was selected. If multiple polyps of equal size were present, observer error was selected over technical error. If multiple polyps with the same size and error type were present, a histologic finding of cancer or adenoma was chosen over other histologic variants. In addition, the primary aim of our study was to determine the effect that various sources of error have on the sensitivity of CT colonography; thus, hypothetical sensitivities were calculated after each source of error was eliminated on a per-patient, per-polyp, and per–histologic finding (adenomas and cancers) basis. Per-patient sensitivities at the 6.0–9.9 mm polyp size range were not determined because patients with multiple polyps (whether larger or smaller than 10.0 mm) would have appeared twice in such an analysis. We did not analyze the effect of sources of error on specificity. All statistical analysis was performed with software (Excel 2002; Microsoft, Redmond, Wash).
|
RESULTS
|
|---|
Imaging and study data from the 614 subjects participating in the original study were obtained; 234 polyps of all histologic types 6.0 mm or larger were found in 155 patients. In addition, we calculated the per-polyp and per-patient sensitivity for adenomas and cancers. When only adenomas and cancers were considered, the per-patient sensitivity was about 68% at the 6.0-mm or larger threshold, as opposed to about 55% for polyps of all histologic types (Table 1).
False-Negative Findings
Figure 2 summarizes the study cohort. Five polyps in three patients were excluded from the analysis because complete image studies could not be retrieved. In addition, one patient was found to have two separate lesions, both of which were identified at CT colonography and by consensus. The size of one of the lesions could not be verified; this lesion was therefore excluded. This left a total of 228 polyps in 152 patients for further analysis. One hundred fourteen of the 228 polyps were false-negative and 114 were true-positive findings. Of the 152 patients with a polyp, 83 had at least one lesion identified at CT colonography. This left 69 patients who were considered to have false-negative findings in our per-patient analysis.
Error Classification
Next, error classification at polyp size thresholds of 6.0 mm or larger, 10.0 mm or larger, and between 6.0 and 9.9 mm was determined for each of the 114 polyps that were false-negative findings at CT colonography according to histologic type (Table 2, Table A1 in Appendix). When polyps of all histologic types were considered, 53% (60 of 114) of these false-negative polyps were attributed to observer-related errors, and 26% were attributed to errors classified as technical. Twenty-one percent of the false-negative polyps were undetectable in retrospect or were considered not prospectively diagnosable and were categorized as not reconcilable. The same error classification method was also performed for the 69 patients with false-negative findings at CT colonography (Table 3, Table A2 in Appendix). Similarly, 51% (35 of 69) of these patients were considered to have had false-negative results owing to observer errors; 23%, owing to technical errors; and 26%, owing to errors that were not reconcilable.
At the 10.0-mm threshold, observer error strongly predominated as the reason for missed adenomas and cancers, accounting for 80% (16 of 20) of the false-negative polyps (Appendix).
Hypothetical Calculations
Hypothetical sensitivity calculations were determined given the findings from our retrospective analysis (Table 4). When observer errors were considered true-positive findings, sensitivity for adenomas and cancers 10.0 mm or larger increased to 93% and 91% at per-polyp and per-patient analysis, respectively. Of the 228 polyps analyzed, 147 were adenomas and cancers 6.0 mm or larger, of which 126 (85.7%) were visible in retrospect and deemed prospectively diagnosable. Additional sensitivity calculations were performed with elimination of false-negative findings caused by observer and technical error (Table 4). With this adjustment, the sensitivity of CT colonography for adenomas and cancers 6.0 mm or larger was 95% on a per-polyp basis and 95% on a per-patient basis. Eight (5.4%) of 147 adenomas and cancers at the 6.0 mm or greater threshold were not visible in retrospect after elimination of observer and technical error.
|
DISCUSSION
|
|---|
The sources of false-negative interpretations of CT colonographic studies have been analyzed in some but not all clinical studies of CT colonography (8) and have been generally categorized as technical or observer errors. Analysis of the sources of errors at CT colonography by its nature is retrospective, which introduces potential bias, but such analysis is nevertheless an important and worthwhile effort because of the implications for guiding improvement in the diagnostic accuracy of CT colonography. Observer errors could potentially be diminished by determining the pitfalls of interpretation and improving reader education.
Technical errors are intrinsic to the nature of the examination and are typically related to colon preparation or cleansing and distention. It was anticipated that improvements in CT scanning technology would help reduce some technical errors. Faster scanners have reduced the length of the necessary breath hold from 1 minute in early CT colonography reports to less than 10 seconds when CT colonography is performed with more recently introduced 40–64-section scanners, thus nearly eliminating breathing artifacts. Thinner collimation and reconstruction intervals have improved the conspicuity of subcentimeter polyps, have made the spatial resolution of multiplanar reconstructions nearly as useful for diagnosis as transverse source images, and have resulted in higher-quality 3D reconstructions. This has improved resolution so that some polyps that were not detectable with older technology are now visible and some polyps that were previously visible are seen with better spatial resolution on 2D and 3D images, thus improving the confidence of interpretation. Although some of these advances postdate the completion of the data sets analyzed in this study, the poor sensitivity initially reported cannot be explained on this basis alone, and we sought an explanation. Sensitivity remains disappointing, even when one considers the recalculated per-patient sensitivity for adenomas and cancers of 70% at 10.0 mm or larger and 68% at 6.0 mm or larger on the basis of the original study results.
Our error analysis was performed well after the completion of the study by using software interpretation tools that have improved since the study's completion. Because the quality of the data sets is an overwhelmingly more important factor, which cannot be altered, it is unlikely that the use of more advanced software affected the retrospective analysis. No attempt was made to reread the entire data set for all patients in our cohort. Rather, a two-part retrospective analysis was performed. First, all the available original data were reanalyzed by using the revised size measurement criteria and polyp location reconciliation method detailed above for comparing CT colonographic and optical colonoscopic results. Second, each CT colonographic study containing a false-negative finding at CT colonography was reviewed by a radiologist experienced in CT colonography interpretation to determine the false-negative error classification and prospective diagnosability.
Accurate polyp size measurement is a critical aspect in determining the sensitivity of CT colonography. Furthermore, determining the size cutoff between clinically irrelevant and relevant CT colonographic findings is much debated. Interestingly, in this data set, we found that seven polyps were accurately identified but were undermeasured during initial prospective CT colonographic evaluation. These measurement errors led to false-negative results because of the size discrepancy with the associated colonoscopic findings. Consequently, this factor contributed to lowering the calculated sensitivity of CT colonography. In addition, in clinical practice, polyp measurement is critical for clinical decision making and follow-up recommendations (9). Using a 3D endoluminal display for polyp measurement sometimes enables more accurate measurements of polyps and avoids the undermeasurement associated with 2D measurement (10); however, this may also vary with the particular software used (11).
The second analysis we performed introduced an additional bias; namely, the experienced readers' determination as to the polyp candidate being diagnosable. This is similar to a legal expert viewing a case to determine if a missed lesion violated the legal standard of care. This analysis is inherently retrospective and biased. To minimize bias, we used a sequential unblinding of the experienced reader—first only to the segment in which the polyp was located and only if the experienced reader could not find the lesion did the unblinded researcher reveal the polyp's size and morphology. Last, if the experienced reader could not identify a polyp with certainty, the unblinded researcher displayed his opinion regarding polyp candidates for review. For most polyps, there was clear consensus as to diagnosability, and only for a minority of polyps was reconciliation particularly difficult.
We found that a major contributor to errors at CT colonography was observer perceptual error, while observer measurement error played a smaller role. Observer perceptual error was the single greatest source of error among all error classifications in this study. Of the 147 adenomas and cancers 6.0 mm or larger found in this study, 31 were false-negative findings because of observer perceptual error and were visible in retrospect. Per-patient sensitivity for clinically relevant findings 6.0 mm or larger increased nearly 20% after observer error was eliminated when CT colonographic studies were interpreted. This finding suggests that increasing reader diagnostic performance is a necessary step toward making CT colonography a more viable screening method. One potential approach to improving reader performance is the use of computer-aided detection as a kind of "spell checker" to present the radiologist with polyp candidates for evaluation (12). Additionally, improved training for readers that stresses the pitfalls of interpretation at CT colonography may help increase reader sensitivity (13). Results of one study (14) demonstrated that training nonradiologists to perform a second reading may be an additional means of improving sensitivity. The effect of other reading paradigms, such as newer views of the colon ("virtual pathology") and electronic subtraction combined with a primary 3D reading, requires further investigation and might improve reader sensitivity.
A second but important source of error in this cohort was technical. Technical errors previously reported to be common included lack of distention, artifacts, excessive stool, and the like (15). Some technical errors are inherent to the technique and can never be completely eliminated but could be minimized by optimizing preparation (eliminating retained stool), by using shorter scanning times, by providing careful patient instructions (to avoid breathing artifacts), by devoting careful attention to the scanning range (altering the scan field of view to avoid excluding some of the colon), and by adding a decubitus view when needed (eg, when there is excessive retained fluid). After observer errors and technical problems deemed amenable to correction by consensus of two experienced readers were accounted for, only 5.4% of adenomas and cancers 6.0 mm or larger were not visible on retrospect, resulting in a best-case per-polyp and per-patient CT colonography sensitivity of 94.6% and 94.8%, respectively. These findings underscore the importance of optimal patient preparation and rigorous technique when performing the examination. Unlike our analysis of CT colonography, a similar analysis for double-contrast barium enemas involving a subset of the same patient cohort we studied revealed that technical errors predominated (16).
Limitations of this study primarily included its retrospective nature. Although this is true of any study that analyzes sources of error, the categorization of polyps in retrospect as diagnosable or not diagnosable could have been evaluated prospectively by having multiple readers reread the studies in the entire cohort, which was not feasible. The number of experienced readers making this decision was limited to two radiologists (A.H.D., R.A.H.). The project was performed with only one software visualization package, but there are no data to suggest that this factor would affect the conclusions. Last, three patients were excluded from the analysis because of missing image data.
Our results demonstrate specific causes of error in interpretation at CT colonography. With this information, we hope that efforts in improving the sensitivity of CT colonography can be targeted more effectively, emphasizing the need for careful training and reading methods and supporting the potential for improved sensitivity with computer-aided detection to direct attention to polyp candidates.
|
APPENDIX
|
|---|
Tables A1 and A2 show per-polyp and per-patient error types.
|
ADVANCE IN KNOWLEDGE
|
|---|
- At the 10.0-mm threshold, observer error strongly predominated as the reason for missed adenomas and cancers, accounting for 80% (16 of 20) of the false-negative polyps.
|
IMPLICATION FOR PATIENT CARE
|
|---|
- Radiologists performing CT colonography can minimize technical errors through proper patient preparation and training of technologists performing the examination. Observer errors should be minimized with slow, meticulous reading. Care should be taken to minimize errors related to undermeasuring of polyps.
|
FOOTNOTES
|
|---|
Abbreviations: ACBE = air-contrast barium enema • 3D = three-dimensional • 2D = two-dimensional
Authors stated no financial relationship to disclose.
Author contributions: Guarantors of integrity of entire study, T.D., A.H.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, D.R., D.C.R, A.H.D.; clinical studies, D.R., R.A.H., K.S.; statistical analysis, T.D., D.R., A.H.D.; and manuscript editing, all authors
|
References
|
|---|
- Fenlon HM, Nunes DP, Schroy PC, Barish MA, Clarke PD, Ferrucci JT. A comparison of virtual and conventional colonoscopy for the detection of colorectal polyps. N Engl J Med 1999;341:1496–1503.[Abstract/Free Full Text]
- Yee J, Akerkar GA, Hung RK, Steinauer-Gebauer AM, Wall SD, McQuaid KR. Colorectal neoplasia: performance characteristics of CT colonography for detection in 300 patients. Radiology 2001;219:685–692.[Abstract/Free Full Text]
- Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 2003;349:2191–2200.[Abstract/Free Full Text]
- Laghi A, Iannaccone R, Carbone I, et al. Computed tomographic colonography (virtual colonoscopy): blinded prospective comparisons with conventional colonoscopy for the detection of colorectal neoplasia. Endoscopy 2002;34(6):441–446.[CrossRef][Medline]
- Cotton PB, Durkalski VL, Pineau BC, et al. Computer tomographic colonography (virtual colonoscopy): a multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713–1719.[Abstract/Free Full Text]
- Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003;125(2):311–319.[CrossRef][Medline]
- Rockey DC, Paulson E, Niedzwiecki D, et al. Analysis of air contrast barium enema, computed tomographic colonography, and colonoscopy: prospective comparison. Lancet 2005;365:305–311.[Medline]
- Fidler JL, Fletcher JG, Johnson CD. Understanding interpretive errors in radiologists learning computed tomography colonography. Acad Radiol 2004;11(7):750–756.[CrossRef][Medline]
- Zalis ME, Barish MA, Choi JR, et al. CT Colonography reporting and data system: a consensus proposal. Radiology 2005;236(1):3–9.[Free Full Text]
- Pickhardt PJ, Lee AD, McFarland EG, Taylor AJ. Linear polyp measurement at CT colonography: in vitro and in vivo comparison of two-dimensional and three-dimensional displays. Radiology 2005;236(3):872–878.[Abstract/Free Full Text]
- Young BM, Fletcher JG, Paulsen SR, et al. Polyp measurement at CT colonography: a multi-reader, multi-workstation comparison. AJR Am J Roentgenol 2006;186:119–121.[Free Full Text]
- Yoshida H, Dachman AH. CAD techniques, challenges, and controversies in computed tomographic colonography. Abdom Imaging 2005;30(1):26–41.[CrossRef][Medline]
- Fletcher JG, Johnson CD, Toledano A, et al. ACRIN 6664: lessons for CT colonography training and certification [abstr]. In: Radiological Society of North America Scientific Assembly and Annual Meeting Program. Oak Brook, Ill: Radiological Society of North America, 2005;470.:
- Bodily KD, Fletcher JG, Engelby T, et al. Nonradiologists as second readers for intraluminal findings at CT colonography. Acad Radiol 2005;12(1):67–73.[CrossRef][Medline]
- Fletcher JG, Johnson CD, MacCarthy RL, Welch TJ, Reed JE, Hara AK. CT colonography: potential pitfalls and problem-solving techniques. AJR Am J Roentgenol 1999;172(5):1271–1278.[Free Full Text]
- Thompson WM, Foster WL, Paulson EK, et al. Causes of errors in polyp detection at air-contrast barium enema examination. Radiology 2006;239(1):139–148.[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
P. J. Pickhardt and A. H. Dachman
Missed Lesions at Primary 2D CT Colonography: Further Support for 3D Polyp Detection
Radiology,
February 1, 2008;
246(2):
648 - 649.
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. J. Pickhardt, A. D. Lee, A. J. Taylor, S. J. Michel, T. C. Winter, A. Shadid, R. J. Meiners, P. J. Chase, J. L. Hinshaw, J. G. Williams, et al.
Primary 2D Versus Primary 3D Polyp Detection at Screening CT Colonography
Am. J. Roentgenol.,
December 1, 2007;
189(6):
1451 - 1456.
[Abstract]
[Full Text]
[PDF]
|
|
|
TOP
ABSTRACT
INTRODUCTION
