Published online before print November 14, 2006, 10.1148/radiol.2421052068
(Radiology 2007;242:120-128.)
© RSNA, 2006
CT Colonography: Automated Measurement of Colonic Polyps Compared with Manual Techniques—Human in Vitro Study1
Stuart A. Taylor, MB, BS, BSc, MD, MRCP, FRCR,
Andrew Slater, MB, BS, MRCP, FRCR,
Steve Halligan, MB, BS, MD, MRCP, FRCR,
Lesley Honeyfield, DCR,
Mary E. Roddie, MB, BS, MRCR, FRCR,
Jamshid Demeshski, PhD,
Hamdam Amin, PhD and
David Burling, MB, BS, MRCP, FRCR
1 From the Department of Imaging, University College Hospital, 2F Podium, 235 Euston Rd, London NW1 2BU, England (S.A.T., S.H.); Department of Imaging, John Radcliffe Hospital, Oxford, England (A.S.); Medicsight, London, England (L.H., J.D., H.A.); Charing Cross Hospital, London, England (M.E.R.); and St Mark's and Northwick Park Hospitals, London, England (D.B.). From the 2005 RSNA Annual Meeting. Received December 19, 2005; revision requested February 13, 2006; revision received February 17; final version accepted April 7.
Address correspondence to S.A.T. (e-mail: csytaylor{at}yahoo.co.uk).
 |
ABSTRACT
|
|---|
Purpose: To prospectively investigate the relative accuracy and reproducibility of manual and automated computer software measurements by using polyps of known size in a human colectomy specimen.
Materials and Methods: Institutional review board approval was obtained for the study; written consent for use of the surgical specimen was obtained. A colectomy specimen containing 27 polyps from a 16-year-old male patient with familial adenomatous polyposis was insufflated, submerged in a container with solution, and scanned at four-section multi–detector row computed tomography (CT). A histopathologist measured the maximum dimension of all polyps in the opened specimen. Digital photographs and line drawings were produced to aid CT–histologic measurement correlation. A novice (radiographic technician) and an experienced (radiologist) observer independently estimated polyp diameter with three methods: manual two-dimensional (2D) and manual three-dimensional (3D) measurement with software calipers and automated measurement with software (automatic). Data were analyzed with paired t tests and Bland-Altman limits of agreement.
Results: Seven polyps (
6-mm diameter) could not be extracted by using the software; 20 polyps (5–15-mm diameter) remained for analysis. Automated measurement was not significantly different from histologic size for the experienced reader (mean difference, 0.63 mm; P = .06) or novice reader (mean difference, 0.58 mm; P = .12). With manual 2D measurement and manual 3D measurement, the experienced reader (1.21-mm mean difference, P < .001, and 0.68-mm mean difference, P = .03, respectively) and novice reader (1.54-mm mean difference, P < .001, and 0.84-mm mean difference, P = .002, respectively) significantly underestimated polyp size. Interobserver agreement was good and similar for all three methods (95% limits of agreement span, approximately 2.5 mm). Intraobserver agreement was related to reader experience, with differences of up to 2.5 mm within expected limits of agreement.
Conclusion: For polyps smaller than 1 cm, measurement differences of up to 2.5 mm are within the expected limits of inter- and intraobserver agreement for all measurement techniques. Automated and manual 3D polyp measurements are more accurate than manual 2D measurements.
© RSNA, 2006
|
INTRODUCTION
|
|---|
It is widely believed that the majority of colorectal cancers arise from precursor adenomatous polyps and that endoscopic polypectomy is probably preventive (1,2). The risk of developing cancer in any individual polyp is strongly correlated with size; less than 1% of polyps smaller than 1 cm in diameter harbor malignancy (3). Furthermore, the majority of polyps do not become cancerous, and there is evidence that some small polyps may even regress (4). Endoscopic polypectomy is not without risk, notably bleeding, perforation, and adverse events related to sedation; complete endoscopic polyp clearance, irrespective of size, is therefore controversial (5,6).
Computed tomographic (CT) colonography is increasingly used for colorectal cancer screening (7). Agreement about clinical treatment strategies for polyps detected by using CT colonography is fundamental to the success of such screening programs, and while few would argue that patients with polyps of 1 cm or larger in diameter should undergo polypectomy, the treatment of smaller polyps is uncertain. Recently, for example, follow-up with CT rather than immediate endoscopic removal has been advocated for polyps that are 6–9 mm in diameter (8). Such treatment strategies rely heavily on the accuracy and reproducibility of polyp size measurement by using CT colonography. Data suggest wide intra- and interobserver variation, however, in diameter measurements determined with CT and a high dependency for accuracy in measurement on the visualization technique used (9,10). Polyp measurement with automated software, which allows elimination of subjectivity, holds considerable promise for reducing inter- and intraobserver variability and for potentially improving accuracy (11,12). To date, the accuracy of measurement with automated software is largely unproved. The purpose of our study was to prospectively investigate the relative accuracy and reproducibility of manual and automated computer software measurements by using polyps of known size in a human colectomy specimen.
|
MATERIALS AND METHODS
|
|---|
The automated measurement software for this study was provided by Medicsight (London, England). Some authors (S.A.T., S.H., M.E.R., L.H., H.A., J.D., D.B.) are employees of or consultants for this company. Authors were aware of industry support. The author who was not an employee of or consultant for Medicsight had full control of the data and information included in this study. Institutional review board approval was obtained for the study. A 16-year-old male patient with familial adenomatous polyposis coli who underwent elective colectomy gave written consent for use of his specimen. Parental consent also was obtained.
Specimen Preparation
At surgery, standard subtotal colectomy was performed, the specimen was washed with 0.9% saline until all fecal residue was removed, and the ileal remnant was tied off with 2/0 polypropylene sutures (Prolene; Ethicon, Edinburgh, Scotland) by the attending surgeon. An 18-F Foley urinary catheter was inserted through the distal lumen, and the retention balloon was fully inflated with 10 mL of water prior to suturing. The specimen was then immediately transferred to the CT suite, and approximately 40 puffs of air were introduced by means of a standard air insufflation bulb attached to the Foley catheter until the specimen was visibly well distended (cecal diameter, >5 cm). We were careful to avoid overdistention and effacement of haustral folds. The inflated specimen was arranged on a plastic board so as to mimic the natural in vivo colonic arrangement and then secured with loosely applied elastic bands. The board and secured specimen were then submerged in a plastic container that held 20 L of 0.9% saline. The saline was mixed with 60 mL of diatrizoate meglumine (Gastrografin; Schering Health Care, Burgess Hill, West Sussex, England), 370 mg of iodine per milliliter, to provide a mean attenuation value of 35 HU (standard deviation, 3) (13).
CT Scanning
The specimen was placed in the CT scanner so as to mimic the normal anatomic supine position, and scanning was performed by using a four-section multi–detector row CT scanner (LightSpeed Plus; GE Medical Systems, Milwaukee, Wis) with a collimation of 1.25 mm, tube current of 50 mA, and pitch of 1.5. Data were reconstructed to provide a 50% overlap relative to section collimation (viewed at 0.625-mm reconstruction interval).
Pathologic Examination
After CT, the specimen was examined immediately by a gastrointestinal histopathologist, who had 15 years of experience in gastrointestinal pathology, in the presence of the study coordinator (S.A.T.), who is a radiologist with experience in more than 600 CT colonographic studies with endoscopic correlation. The unfixed specimen was opened along the antimesenteric border and pinned to a dissection board before it was divided into 15-cm-long sections (marked with colored pins) from the sigmoid end. Polyps in each section were measured in situ by using a handheld magnifier and fine measurement calipers and were classified as pedunculated or sessile. In the case of irregular polyps, the largest diameter of the polyp was used, and only polyps that were 3 mm or larger in diameter were documented. The study coordinator took high-resolution digital photographs of each 15-cm-long section and, additionally, drew freehand "maps" that detailed the relative position of each polyp within the segment. On the maps, the study coordinator labeled distances from the segment boundary to neighboring polyps and related these relative positions to recognizable landmarks such as haustral folds, the ileocecal valve, etc. The coordinator also noted uniquely shaped polyp clusters or individual large polyps to aid correlation. There were a total of 27 polyps (3–15 mm in diameter) in the specimen.
CT Annotation
The study coordinator examined the data set at a CT colonographic workstation that allowed analysis with both multiplanar two-dimensional (2D) reformations (Navigator 2; GE Medical Systems) and endoluminal three-dimensional (3D) surface-rendered images (Advantage Windows 4.0; GE Medical Systems). With the images and data, the study coordinator located visible polyps and cross referenced them with the digital photographs and hand-drawn maps of each 15-cm-long colonic segment. Polyp-to-polyp correlation was achieved by using the data recorded on the maps of the specimen.
Once an individual polyp was positively identified in the data set, an arrow was inserted to indicate its location, an annotation identified with a number that corresponded to the polyp was entered, the appropriate section number was recorded, and a screen-save image capture was made.
Image Analysis
Two readers analyzed the data set: One (A.S.) was a radiologist who was experienced in more than 200 CT colonographic examinations with endoscopic correlation, and the other (L.H.) was a radiographic technician who had documented adequate performance and had a detection rate of greater than 70% for polyps of 6 mm or larger in diameter in a test data set of 40 endoscopically verified cases after 1 month of training. Both readers were provided with the exact location of all polyps identified in the specimen through the screen-save image capture on printed documentation and were instructed to measure the maximum polyp diameter, as outlined later.
Manual Measurement
The CT data were downloaded onto the same workstation as was mentioned before. Readers were instructed to measure the maximum 2D polyp diameter by using electronic calipers embedded in the software on a colonic viewing window (window level, –150 HU; window width, 1500 HU) and were free to use multiplanar reformations (including oblique views) to obtain a maximum dimension. Readers were encouraged to magnify the image prior to measurement.
In addition, readers were instructed to measure the maximum 3D polyp diameter on the surface-shaded endoluminal rendered image by using electronic calipers embedded within the software. Readers were free to navigate through the endoluminal rendered image to obtain the optimum view for measurement.
Automated Measurement
The CT data set was downloaded onto a stand-alone workstation equipped with developmental CT colonographic viewing software (Colon CAR 1.3; Medicsight). The software included an automated polyp extraction and measurement function. To use this function, readers were instructed to place two software seed points opposite each other at the perceived junction between the polyp and the colonic wall on the single magnified 2D transverse image they thought best showed the polyp (Fig 1). In brief, the software automatically repositioned the subjectively placed seed points on a calculated polyp boundary.
View larger version (81K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1a: Transverse images show placement of seed points by using automated measurement software. (a) A 6-mm-diameter sigmoid polyp (arrow) well seen within colectomy specimen. (b) Two software seed points that the readers placed opposite each other at the perceived junction between polyp and colonic wall prior to automated extraction and measurement by using software.
|
|
View larger version (76K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1b: Transverse images show placement of seed points by using automated measurement software. (a) A 6-mm-diameter sigmoid polyp (arrow) well seen within colectomy specimen. (b) Two software seed points that the readers placed opposite each other at the perceived junction between polyp and colonic wall prior to automated extraction and measurement by using software.
|
|
The polyp was then segmented by using a region-growing scheme that was based on fuzzy logic to distinguish the polyp from the surrounding normal structures. The boundary of the segmented polyp was indicated to the user with a line that was superimposed on 2D transverse images (Fig 2), together with the calculated maximum diameter, which readers then documented on their report forms. If the readers thought that segmentation of a particular polyp was not possible with the software or if they thought that, with automated segmentation, overestimation of the polyp diameter (eg, with extraction of an adjacent fold) had occurred, they were instructed to manually replace the seed points up to a maximum of three times. If accurate segmentation was still not possible, readers marked that the extraction failed with the software and recorded, in their opinion, the reason for the failure. In particular, readers were asked to note if those polyps for which extraction failed were positioned on a haustral fold or appeared flat (defined as a height of less than 3 mm).
Measurement Order
For each polyp in the data set, manual measurement method order (2D or 3D first) was randomized for each reader. Readers received strict instruction to document their measurements immediately after they were performed, although they were not blinded to their own measurements. All manual and automated polyp measurements were performed twice, 1 week apart, to assess intraobserver variation. At the first session, readers performed manual measurements followed by automated measurement. The study sheet was handed to the study coordinator after completion of manual measurements, prior to commencing automated analysis. The reading order was reversed for the second reading session. The order in which polyps were measured was randomized between manual and automated measurements and between the two readings.
Statistical Analysis
Statistical analysis was performed by using a commercially available statistical software program (StatsDirect, version 2.5.5, 2006; StatsDirect, Cheshire, England). Measurement accuracy was assessed by comparing the mean of the readers' two measurements with the known histologic reference diameter for each measurement method by using the paired t test (a parametric test was used because data were observed to be normally distributed). The level of agreement between each reader's measurement and the true histologic diameter was then examined by calculating the Bland-Altman limits of agreement (again by using the mean of their two measurements). The mean difference was used to measure the size of the range of differences, expressed as 95% limits of agreement, between the measurements that were likely to occur (14) and was calculated as follows: Mean difference = (actual measurement minus observed measurement) ± 1.96 times standard deviation of differences.
The Bland-Altman limits of agreement were then calculated for the paired measurements from each observer (to estimate intraobserver variation) and between observers for each method (to estimate interobserver agreement). For the latter comparison, the average of the two values for each polyp from each observer for each method was used. A difference with P < .05 was considered significant.
|
RESULTS
|
|---|
Clinical Data
A total of six polyps (3–6 mm in diameter) could not be extracted by at least one of the readers by using the automated software. In all cases, readers stated that the polyp appeared flat. Automated measurement of a further 5-mm-diameter polyp was rejected by both readers because of additional extraction of the adjacent haustral fold. The diameters of the remaining 20 polyps were as follows: 5 mm, eight polyps; 6 mm, six polyps; 7 mm, three polyps; 8 mm, two polyps; and 15 mm, one polyp. All polyps were sessile.
Method Accuracy
Readers tended to underestimate polyp diameter in the 20 polyps: For the radiologist, the measurement was smaller than the actual polyp diameter in 14 polyps with the automated method, in 18 polyps with the manual 2D method, and in 14 polyps with the manual 3D method. For the radiographic technician, the measurement was smaller than the actual polyp diameter in 12 polyps with the automated method, in 18 polyps with the manual 2D method, and in 16 polyps with the manual 3D method. For both readers, manual 2D and 3D (but not automated) measurements both differed significantly from the actual polyp diameter (Table 1). The mean difference in measurements was larger for manual 2D compared with manual 3D methods. The span of 95% limits of agreement (approximately 4–5 mm) was similar for all three measurement methods, with no clear association with polyp diameter.
View this table:
[in this window]
[in a new window]
|
Table 1. Mean Difference and Bland-Altman 95% Limits of Agreement for Measured and Absolute Polyp Size according to Measurement Method
|
|
Polyp Categorization
Most polyps of 5 mm or smaller in diameter were correctly classified by both readers by using all three measurement methods (Table 2). Many polyps of 6–9 mm in diameter, however, were misclassified as having a size of 1–5 mm in diameter, particularly by using manual 2D measurement. Automated and manual 3D measurements were similar in classification accuracy. Of the three 7-mm-diameter polyps and two 8-mm-diameter polyps, two 7-mm-diameter polyps and one 8-mm-diameter polyp were correctly classified by both readers as 6–9 mm in diameter by using automated and manual 3D measurement. Neither reader estimated the diameter of any subcentimeter polyp to be larger than 1 cm by using any of the three measurement methods.
Interobserver Agreement
There was no significant difference between measurements made by the two readers for any of the three methods used; the mean difference was 0.29 mm or less (Table 3). Bland-Altman plots with 95% limits of agreement are shown for each measurement method (Figs 3–5). Interobserver agreement was good and similar for all methods, and the span of the 95% confidence interval was approximately 2.5 mm. Analysis of the Bland-Altman plots (Figs 3–5), however, revealed that, for most polyps, agreement tended to be better for measurements with the automated software (differences for interobserver measurements were within 0.6 mm for all but three polyps). The largest difference for interobserver measurements with the automated software was 2.6 mm (radiologist measurement, 7.3 mm; radiographic technician measurement, 4.7 mm). The histologic measurement for the polyp was 5 mm in diameter, a measurement that suggested that the radiologist had incorrectly accepted an overestimate of the polyp boundary determined by using the software.
View this table:
[in this window]
[in a new window]
|
Table 3. Mean Difference and Bland-Altman 95% Limits of Agreement for Interobserver Comparisons of Polyp Measurement according to Measurement Method
|
|
View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 3: Graph shows interobserver agreement by using automated measurement software. Bland-Altman plot of difference between polyp measurements of radiologist and technician against mean measurement, with 95% limits of agreement. For most polyps, agreement was good with the automated software; differences between measurements of readers were within 0.6 mm for all but three polyps.
|
|
View larger version (20K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4: Graph shows interobserver agreement for measurement by using manual 3D method. Bland-Altman plot of difference between polyp measurements of radiologist and technician against mean measurement, with 95% limits of agreement. In general, the scatter of measurement differences around the mean was greater than that with the automated software, although overall 95% limits of agreement were similar.
|
|
View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5: Graph shows interobserver agreement by using manual 2D measurement. Bland-Altman plot of difference between polyp measurements of radiologist and technician against mean measurement, with 95% limits of agreement. In general, scatter of measurement differences around mean was greater than that with either manual 3D measurement or automated software, although overall 95% limits of agreement were similar.
|
|
Intraobserver Agreement
There was no significant difference between the first and second measurements of polyp diameter for any of the three methods for either reader (Table 4). Examples of Bland-Altman plots are shown for each measurement method for both readers (Figs 6–8). For the radiologist, intraobserver agreement was best for the automated software measurement (95% confidence interval span, approximately 3 mm) and worst for the manual 2D measurement (95% confidence interval span, approximately 5 mm). The first and second readings were identical in 17 (85%) of 20 measurements of polyp diameter (to the nearest whole millimeter) with the automated software, but they were identical in 11 (55%) of the measurements with manual 2D measurement and in 13 (65%) with manual 3D measurement.
View this table:
[in this window]
[in a new window]
|
Table 4. Mean Difference and Bland-Altman 95% Limits of Agreement for Intraobserver Comparisons of Polyp Measurement according to Measurement Method
|
|
View larger version (22K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 6: Graph shows intraobserver agreement by using automated measurement software. Bland-Altman plot of difference between polyp size measurement for readings 1 and 2 against mean measurement, with 95% limits of agreement (both readers). In general, intraobserver agreement was very good for all but two polyps. CAR = computer-assisted reader.
|
|
View larger version (23K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 7: Graph shows intraobserver agreement by using manual 2D measurement. Bland-Altman plot of difference between polyp size measurement for readings 1 and 2 against mean measurement, with 95% limits of agreement (both readers). In general, scatter of measurement differences around mean was greater than that with automated software.
|
|
View larger version (24K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 8: Graph shows intraobserver agreement by using manual 3D measurement. Bland-Altman plot of difference between polyp size measurement for readings 1 and 2 against mean measurement, with 95% limits of agreement (both readers). In general, scatter of measurement differences around mean was greater than that with automated software.
|
|
Conversely, intraobserver agreement for the radiographic technician was best for both automated and manual 2D measurements (95% confidence interval span, approximately 3 mm) and was worst for manual 3D measurement (95% confidence interval span, approximately 4 mm). The first and second readings were identical in 16 (80%) measurements of polyp diameter (to the nearest whole millimeter) with the automated software, but they were identical in seven (35%) of the measurements with manual 2D measurement and in eight (40%) with manual 3D measurement.
|
DISCUSSION
|
|---|
Investigators in previous comparative studies of CT colonographic polyp measurements have used endoscopic assessment of polyp size as a reference, but endoscopic polyp size is known to be inherently inaccurate (15,16). To our knowledge, ours is the first study in which human polyps of precisely known size were used.
On the basis of analysis with a paired t test, we found evidence that, regardless of reader experience, manual 3D and automated measurements were on average more accurate than manual 2D measurements. The results are also consistent with those of a recent study that suggest that manual 3D polyp measurement is more accurate than manual 2D measurement (10). Bland-Altman 95% limits of agreement, however, often are regarded as a more robust method for assessment of agreement between the measured and the actual size, and in this regard, all three methods were similar. In a clinical context, the results suggest that, for the polyp population tested, estimates of size determined by using any of the three methods will lie no more than 3 mm away from the true value 95% of the time. Although this finding may seem disappointing in terms of CT colonographic measurement accuracy, it should be remembered that the Bland-Altman test of agreement is a relatively exacting statistic.
Both readers tended to underestimate polyp size. Most polyps of 1–5 mm in diameter were thus on the whole correctly classified by both readers by using all three measurement techniques. Many 6–9-mm-diameter polyps, particularly one of 7 mm and one of 8 mm, however, often were misclassified as having a diameter of 1–5 mm. Classification that was based on manual 3D and automated measurements was equivalent and superior to that based on manual 2D measurements. Such misclassification may have important implications. The recommendations of previous researchers suggest that we ignore polyps that are smaller than 5 mm in diameter but report lesions that are 6 mm or larger in diameter and follow up the latter with either CT colonography or endoscopic removal, depending on age and comorbidity of the patient (8,17).
Our data suggest that larger lesions can be misclassified and potentially ignored. Clearly, there is a fine line between overestimating a measurement, which may prompt unnecessary colonoscopy, and missing potentially clinically important lesions, but our data suggest care must be taken in regard to those lesions that are clearly nondiminutive but still smaller than 1 cm in measured size. Reassuringly, the single 15-mm-diameter polyp was always correctly classified, although one polyp is insufficient to extrapolate our results for all lesions 10 mm or larger in diameter. Many polyps were 6 mm in diameter (ie, on the classification), and the specimen was from a young patient with familial adenomatous polyposis; as such, many of the polyps were relatively flat, although we did not quantify this finding in any detail. The experiment was thus a very difficult test of reader measurement and classification.
Interobserver agreement was relatively good for all three methods, with the 95% level of agreement spanning approximately 2.5 mm. Results of previous work suggest that interobserver agreement for measurements with the automated technique is superior to agreement for those with manual techniques used to measure artificial polyps in a plastic phantom (12). Review of the Bland-Altman plots in the present study also revealed that, in general, interobserver agreement was good with real polyps by using the current software; readers' measurements were within 0.6 mm of each other for all but three polyps. The largest discrepancy (2.6 mm) occurred when the reader who was a radiologist accepted an automated measurement of 7.3 mm for a 5-mm-diameter polyp. Further development of the software may reduce the potential for such a discrepancy, but in the meantime, boundary extraction must be carefully checked by the user.
With the software, seven polyps (all 6 mm in diameter) could not be correctly extracted, and manual measurements were, therefore, more robust overall than were automated measurements. As discussed previously, the relatively flat nature of the polyps posed a difficult challenge, and further refinements to the extraction algorithm may reduce this problem. In addition, at least one polyp on a fold was a challenge to the software, and manual measurement may be best for such lesions.
We did not fully evaluate the relationship between software performance and the relationship of polyps to folds, and further study is required. Although one advantage of the software is the ability of the user to replace seed points if extraction is inaccurate, it could be argued that the software tested is therefore not truly automated and requires manual input. The fact that there was any inter- and intraobserver variation at all in measurements with the software suggests that there remains a reliance on manual seed point placement. The software may therefore be best described as "semiautomated," although we prefer to reserve this term for iterations in which the user must manually trace polyp outline before automated software measurement. Although we used 1.25-mm collimation, the influence of submillimeter collimation with newer 64-section CT technology requires further investigation.
Intraobserver agreement differed according to measurement method. When used by the radiologist, the automated software alone was superior, whereas both automated and manual 2D measurements were superior to manual 3D measurements for the radiographic technician. Review of the Bland-Altman plots revealed that intraobserver agreement with use of the software was excellent in general; both readers recorded identical measurements for more polyps with the automatic technique than with either manual technique. As for interobserver agreement, however, the 95% limits of agreement for the software were broadened by a relatively wider discrepancy from a single polyp measurement.
Our data have potentially important clinical implications. The principle behind CT colonographic follow-up of smaller polyps is that interval growth can be recognized such that endoscopic removal may be advised. Our study reproduced the clinical scenario whereby a patient with multiple small polyps presents again for follow-up with no interval change in polyp size, with the CT colonographic images being read by either a different observer or the same observer. The results are reassuring in that, in general, both intraobserver agreement and interobserver agreement were reasonable for both manual and automated methods. It should be recognized, however, that on the basis of these results, apparent polyp "growth" of less than 2.5 mm cannot be assumed to be actual growth and is within the expected level of inter- and intraobserver variation. Although manual 3D measurement was more accurate than manual 2D measurement for actual size measurement, this was not true for measurements of agreement.
In terms of future clinical recommendations, accuracy of polyp measurement is superior by using either manual 3D measurement or automated software, if available. Current iterations of measurement software, however, are such that the reader should check all extractions for accuracy; some polyps, particularly those that are flat or located on a fold, may not be extracted at all. In terms of inter- and intraobserver agreement, there was no clear advantage for any of the three measurement techniques. For most polyps, however, interobserver agreement and intraobserver agreement were very good by using automated software, and assuming satisfactory extraction on both CT data sets, the software should provide reliable data as to polyp growth. If the software is not available or extraction is poor, either manual 2D or manual 3D measurement has similar limits of agreement, but perhaps manual 3D measurement is favored because of superior intrinsic accuracy.
Our study had limitations. The unavoidable exclusion of polyps that could not be extracted by using the software may have skewed the results in its favor and away from manual techniques. Readers were not blinded to their own measurements, but the measurement order (manual 2D or manual 3D technique first) was randomized on a per-polyp basis. We did not specifically address the issue of flat polyps by producing a subset analysis of the data. All but one polyp was smaller than 1 cm, although the range of polyp sizes used for the experiment mimics that currently being advocated for CT colonographic surveillance. Perhaps most important, we used a single representative for each reader category, and these representatives for radiologists and radiographic technicians acted as surrogates for all radiologists and radiographic technicians, respectively. Inevitably, there will be reader-to-reader variation, and care must be taken when one extrapolates our data to the wider radiologic community. Although we used one particular manufacturer's software, others are currently available and await detailed validation. Finally, we did not attempt to measure polyp volume. Such volumetric measurement may prove more accurate in assessment of polyp growth and is clearly a topic for future research.
Practical application: Polyps smaller than 1 cm in diameter may be measured with sufficient accuracy and interobserver agreement to support CT colonographic surveillance, although inter- and intraobserver error of up to 2.5 mm is possible. Automated computer software holds considerable promise as a way of both improving polyp measurement accuracy and reducing observer variation in the future.
|
ADVANCES IN KNOWLEDGE
|
|---|
- For polyps smaller than 1 cm in diameter, the expected measurement error is approximately 3 mm for both manual and automated measurement techniques.
- In general, automated and three-dimensional measurements are more accurate than is two-dimensional measurement for polyp size.
- For polyps smaller than 1 cm in diameter, measurement differences of up to 2.5 mm are within the expected limits of inter- and intraobserver agreement for both manual and automated measurement techniques.
|
FOOTNOTES
|
|---|
Abbreviations: 3D = three-dimensional • 2D = two-dimensional
See Materials and Methods for pertinent disclosures.
Author contributions: Guarantor of integrity of entire study, S.A.T.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, S.A.T., S.H., J.D.; clinical studies, S.A.T., A.S., M.E.R., H.A., D.B.; experimental studies, S.A.T., A.S., L.H., J.D., H.A., D.B.; statistical analysis, S.A.T., S.H.; and manuscript editing, S.A.T., A.S., S.H., L.H., M.E.R., J.D., D.B.
|
References
|
|---|
- Citarda F, Tomaselli G, Capocaccia R, Barcherini S, Crespi M. Efficacy in standard clinical practice of colonoscopic polypectomy in reducing colorectal cancer incidence. Gut 2001;48:812–815.[Abstract/Free Full Text]
- Winawer SJ, Zauber AG, Ho MN, et al. Prevention of colorectal cancer by colonoscopic polypectomy. The National Polyp Study Workgroup. N Engl J Med 1993;329:1977–1981.
- Muto T, Bussey HJ, Morson BC. The evolution of cancer of the colon and rectum. Cancer 1975;36:2251–2270.[Medline]
- Hofstad B, Vatn MH, Andersen SN, et al. Growth of colorectal polyps: redetection and evaluation of unresected polyps for a period of 3 years. Gut 1996;39:449–456.[Abstract/Free Full Text]
- Ferrucci JT. Colon cancer screening with virtual colonoscopy: promise, polyps, politics. AJR Am J Roentgenol 2001;177:975–988.[Free Full Text]
- Ransohoff DF. CON: immediate colonoscopy is not necessary in patients who have polyps smaller than 1 cm on computed tomographic colonography. Am J Gastroenterol 2005;100:1905–1907.[CrossRef][Medline]
- Pickhardt PJ, Taylor AJ, Johnson GL, et al. Building a CT colonography program: necessary ingredients for reimbursement and clinical success. Radiology 2005;235:17–20.[Free Full Text]
- Zalis ME, Barish MA, Choi JR, et al. CT colonography reporting and data system: a consensus proposal [editorial]. Radiology 2005;236:3–9.[Free Full Text]
- Burling D, Halligan S, Taylor SA, et al. Polyp measurement using CT colonography: agreement with colonoscopy and effect of viewing conditions on interobserver and intraobserver agreement. AJR Am J Roentgenol 2006;186:1597–1604.[Abstract/Free Full Text]
- Pickhardt PJ, Lee AD, McFarland EG, Taylor AJ. Linear polyp measurement at CT colonography: in vitro and in vivo comparison of two-dimensional and three-dimensional displays. Radiology 2005;236:872–878.[Abstract/Free Full Text]
- Blake ME, Soto JA, Hayes RA, Ferrucci JT. Automated volumetry at CT colonography: a phantom study. Acad Radiol 2005;12:608–613.[CrossRef][Medline]
- Burling D, Halligan S, Roddie ME, et al. Computed tomography colonography: automated diameter and volume measurement of colonic polyps compared with a manual technique—in vitro study. J Comput Assist Tomogr 2005;29:387–393.[CrossRef][Medline]
- Dachman AH, Lieberman J, Osnis RB, et al. Small simulated polyps in pig colon: sensitivity of CT virtual colography. Radiology 1997;203:427–430.[Abstract/Free Full Text]
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–310.[CrossRef][Medline]
- Fennerty MB, Davidson J, Emerson SS, Sampliner RE, Hixson LJ, Garewal HS. Are endoscopic measurements of colonic polyps reliable? Am J Gastroenterol 1993;88:496–500.[Medline]
- Schoen RE, Gerber LD, Margulies C. The pathologic measurement of polyp size is preferable to the endoscopic estimate. Gastrointest Endosc 1997;46:492–496.[CrossRef][Medline]
- Macari M, Bini EJ, Jacobs SL, et al. The significance of missed polyps at CT colonography. AJR Am J Roentgenol 2004;183:127–134.[Abstract/Free Full Text]
TOP
ABSTRACT
INTRODUCTION
