|
|
||||||||
Cardiac Imaging |
1 From the Department of Epidemiology, University of Michigan, Ann Arbor (L.F.B., P.A.P.); and the Department of Diagnostic Radiology, Mayo Clinic and Foundation, 200 Second St SW, Rochester, MN 55905 (P.F.S.). Received February 7, 2000; revision requested April 3; revision received June 19; accepted July 14. Supported by National Institutes of Health grant R01 HL46292 and a General Clinical Research Center grant from the National Institutes of Health (MO1-RR00585) awarded to Mayo Clinic Rochester. Address correspondence to P.F.S. (e-mail: psheedy@mayo.edu).
| ABSTRACT |
|---|
|
|
|---|
MATERIALS AND METHODS: A total of 1,376 asymptomatic research participants, who were not selected because they were at high risk for coronary artery disease, were examined for the quantity of CAC with dual scan runs at electron-beam CT. With these data, 95% limits of agreement were established and used to evaluate differences between scan runs performed approximately 3.5 years apart in 81 participants.
RESULTS: The 95% limits of agreement depended on the mean quantity of CAC in the dual scan runs. Of the 81 participants whose examinations were approximately 3.5 years apart, 59 (73%) had no apparent change in CAC between the two examinations, 21 (26%) had large increases suggesting progression of CAC, and one (1%) had a large decrease suggesting regression of CAC.
CONCLUSION: The demonstrated method can be used to evaluate both agreement in dual scan runs and change in quantity of CAC over time.
Index terms: Computed tomography (CT), electron beam, 54.12111, 54.12119 Coronary vessels, calcification, 54.81 Coronary vessels, CT, 54.12111, 54.12119 Statistical analysis
| INTRODUCTION |
|---|
|
|
|---|
In studies (2,916) of the repeatability of electron-beam CT measurements of CAC, participants were examined with dual scan runs (acquisition of two sets of scans) within a short period. Many researchers, including ourselves, have used a variety of methods to evaluate the repeatability of such measures. These methods include use of Pearson product moment correlation coefficients, intraclass correlation coefficients, and linear regression and comparisons of the mean quantity of CAC with the first scan run and the mean with the second scan run (2,916). For reasons stated by Bland and Altman (21), these approaches are "misleading" or "inappropriate" in studies in which agreement is assessed.
Comparison of results from electron-beam CT and dual scan runs is difficult. The range of arithmetic differences (scan 1 minus scan 2) in the quantity of CAC between the dual scan runs increases with the mean quantity of CAC for the dual scan runs. Also, the range of relative differences ([scan 1 minus scan 2] divided by the mean of scan 1 and scan 2) decreases with the mean of the dual scan run results (2,916). Thus, the magnitude of the arithmetic differences and relative differences must be interpreted differently, depending on the mean quantity of CAC.
In 1986, Bland and Altman (22) described a method to compare two measurement results by plotting the arithmetic differences versus the mean and by calculating limits of agreement for the differences. This method has been widely used and reported in the medical literature and has the critical assumption that the magnitude of the differences is independent of the magnitude of the mean (ie, the differences are uniform over the observed range of the mean) (2123). This assumption is violated with use of the quantity of CAC, even after logarithmic transformation.
Recently, Bland and Altman (21) extended their method with a regression approach for nonuniform differences when the relationship between the difference and the mean is complicated and when use of a logarithmic transformation does not result in an uniform distribution of the differences over the range of the mean. This regression approach is appropriate for the comparison of electron-beam CT results and, to our knowledge, has not been previously used in this application.
The purposes of the present study were (a) to apply the regression approach for nonuniform differences to quantity-of-CAC results in a large group of individuals recruited from a general asymptomatic population, (b) to demonstrate how to incorporate this approach into an ongoing quality-control program, and (c) to illustrate the use of this approach to evaluate changes in the quantity of CAC over time in an individual.
| MATERIALS AND METHODS |
|---|
|
|
|---|
A subset of 81 participants in the present study were examined as part of a pilot study (19) to assess changes in CAC over time. These 81 participants were examined with a single scan run at baseline between March 1, 1990, and September 30, 1992, and two consecutive scan runs performed minutes apart at follow-up between November 1, 1994, and August 1, 1995. Single scan runs were used at baseline because participants underwent scanning prior to our having knowledge of the repeatability issues with electron-beam CT. The mean period between baseline and follow-up was 3.5 years ± 0.4 (SD) (2).
At baseline and follow-up, none of these participants had a history of definite angina, heart surgery, or myocardial infarction. They were informed by letter of the results of their baseline and follow-up examinations. This letter included information about their risk factor status and results of electron-beam CT. Information was not collected at follow-up to allow us to determine whether participants changed their health behavior because of the results of their baseline examinations.
The remaining participants (n = 1,311) underwent baseline examination of CAC that consisted of two consecutive scan runs performed minutes apart between December 1, 1992, and April 30, 1999. These 1,311 participants did not undergo follow-up.
All electron-beam CT performed prior to October 1, 1996, was conducted with a C-100 scanner (Imatron, South San Francisco, Calif). After October 1, 1996, all electron-beam CT was conducted with a C-150 scanner (Imatron).
The scanning protocol has been described in detail elsewhere (26). A scan run consisted of the acquisition of 40 contiguous transverse two-dimensional images of 3-mm-thick sections at the level above the coronary artery origins to the cardiac apex. Exposure duration was 100 msec per tomographic level. Approximately 3540 contiguous images of 3-mm-thick sections, with 3-mm table incrementation, were obtained with each acquisition. Images were acquired with electrocardiographic triggering at 80% of the R-R interval. Images were obtained by using a 26- or 30-cm2 field of view and a 512 x 512 reconstruction matrix so that the area represented by 1 pixel was 0.26 or 0.34 mm2, respectively. The same field of view was used with each scan run of the dual scan runs. Radiation exposure was 10 mGy (1 rad) per image. No contrast agent was used. The dual scan runs were performed minutes apart. After the first scan run, participants sat up and moved about briefly.
Image Analysis
One of two radiologists (P.F.S.) blinded to all participant information inspected the sequential images from each scan run and compared them with each other. In this way, it was possible to qualitatively assess whether there was an observable technical problem with the scan run, such as an obvious gap or overlap between adjacent tomograms. Errors in triggering were detected by using the scanner software. Baseline scan runs in 16 participants were clearly technically flawed due to obviously visible gaps, overlaps, or errors in triggering, and results of these were excluded from the analyses. The final sample consisted of 1,376 participants (680 women, 696 men). The mean age was 56.2 years ± 13.5.
Radiologic technologists initially processed the images with automated operator-independent image-processing software (27). Image quality and scoring accuracy of each of the two runs from each examination was assessed by one of the two radiologists, who carefully inspected each image and made vessel-by-vessel and calcific focusbycalcific focus comparisons. Care was taken that no calcific focus was overlooked and that no noncoronary calcific focus (eg, aortic root or valve calcification) was included. Also, any area of artifact or noise adjacent to but not within a vessel was not included in the result.
CAC was defined as a focus of 4 or more adjacent pixels in size within 5 mm of the midline of a coronary artery, with a CT attenuation number above 130 HU for each pixel in the focus. The software was used to measure the area (in square millimeters) of each focus. For each examination, the area of the hyperattenuating foci was summed to calculate the quantity of CAC and was referred to as the calcific area. Although we and others have used other measures of the quantity of CAC, such as the Agatston score (28) and volumetric measures (14), we chose to use the calcific area for two reasons. First, the Agatston score has been criticized (14,29,30) because it depends on an arbitrary weighting coefficient that is used to multiply the area of a focus by a coefficient based on the highest attenuating pixel in a focus. Second, the calcific area is the actual measurement that is obtained at each scanning level and does not depend on a postprocessing regression approach of isotropic interpolation to derive a volume from two-dimensional data (14).
Statistical Analysis
Repeatability of the dual scan runs was assessed in 1,376 participants (81 underwent dual scan runs at follow-up and 1,295 underwent dual scan runs at baseline). For comparison with other study findings of the repeatability of dual scan examinations for CAC (2,916), the McNemar test was used to test for the equality of the prevalence of CAC in scan runs 1 and 2 (31). Calcific area in each scan run was transformed by using the natural log (ln[calcific area + 1]) to reduce skewness. A paired t test was used to test for the equality of the mean transformed calcific area in scan runs 1 and 2. The intraclass correlation coefficient was calculated to estimate the correlation between transformed calcific area in the dual scan runs. The arithmetic and relative differences in nontransformed calcific area between the dual scan runs were plotted across the mean calcific area.
The regression method for nonuniform differences was used to assess the repeatability of examination results (21). Briefly, we used linear regression to model the arithmetic difference D as a function of the mean calcific area A with the following equation: D = b0 + b1A, where b0 is the intercept and b1 is the slope of the linear regression line.
Next, we used linear regression to model the scatter of the absolute value of the residual R from the previous model as a function of the mean calcific area with the following equation: R = c0 + c1A, where c0 is the intercept and c1 is the slope of the linear regression line.
By combining the parameter estimates from both of these models, the 95% limits of agreement were calculated as follows: b0 + b1A ± 2.46(c0 + c1A).
In this equation, 2.46 rather than 1.96 was used because we modeled the absolute value of the residuals rather than the residuals themselves. The absolute value of the residuals will follow a half-normal distribution, and it is necessary to multiply 1.96 by the square root of
/2 (21).
The arithmetic differences and the 95% limits of agreement were plotted against the mean calcific area. By using these limits of agreement, we estimated the calcific area expected at the upper and lower 95% limits of agreement for dual scan runs with a specific mean. Information from these limits was used to evaluate results obtained minutes or years apart.
Quality-Control Program
As part of a quality-control program, the 95% limits of agreement were used to identify dual scan run findings in an individual that are unexpectedly different to determine the reason for the difference. A radiologist (P.F.S.) retrospectively examined dual scan run results in 30 participants with arithmetic differences outside the 95% limits of agreement. These results were scrutinized for partial volume effects; gaps; overlaps; triggering problems or ECG misregistrations; arrhythmias; heart or patient motion; errors in patient positioning or table incrementation; noise; artifact; foci not in an artery; and aortic rim, valvular, or pericardial calcifications.
Change in CAC over Time
With data from the 81 participants who underwent a single scan run at baseline and dual scan runs at follow-up, a scatter plot was made of the mean calcific area in the dual scan runs at follow-up versus the calcific area in the single scan run at baseline. Overlaid on the plot were lines indicating the range of calcific area predicted from the 95% limits of agreement for the 1,376 participants. If the baseline result was within the predicted range of the follow-up results, the participant was considered to have no change in the detectable quantity of CAC. If the baseline result was to the left of the predicted range at follow-up, the participant was considered to have progression in the detectable quantity of CAC. That is, the baseline result was considerably less than the mean in the dual scan runs at follow-up. If the baseline result was to the right of the predicted range at follow-up, the participant was considered to have regression in the detectable quantity of CAC. That is, the baseline result was considerably greater than the mean in the dual scan runs at follow-up.
| RESULTS |
|---|
|
|
|---|
Figure 1 shows a bar chart of the nontransformed calcific area in the scan run with positive findings of 75 participants with CAC in only one of the two scan runs. One-half (n = 37) of these participants had a CAC area of less than 2.0 mm2, and the maximum calcific area in the scan run with positive findings was 12.7 mm2. Consistent with other study findings (2,916), the range of arithmetic differences in nontransformed calcific area between the two consecutive scan runs was larger in participants with a higher mean calcific area compared with those with a lower mean calcific area (Fig 2). Also, the range of relative differences was smaller in participants with a higher mean calcific area compared with those with a lower mean calcific area (Fig 3). As expected, the magnitude of the arithmetic and relative differences must be interpreted differently at different levels of mean calcific area.
|
|
|
|
|
|
|
| DISCUSSION |
|---|
|
|
|---|
The regression approach demonstrated in this study can be used to evaluate agreement in the quantity of CAC between dual scan runs performed minutes apart. As part of our quality-control program, a radiologist retrospectively reexamined dual scan run findings in 30 participants with arithmetic differences that were outside the 95% limits of agreement. Partial volumes (n = 9) were the most common source of large variations. Gaps (n = 6) and overlaps (n = 5) between adjacent tomograms caused some of the large variations, and either could have been due to breathing, arrhythmia, patient motion, heart motion, or table incrementation. Noise or artifact (n = 6) was present but was usually found with both scan runs of the same examination. Aortic rim calcification (n = 4) that was incorrectly labeled as being within a coronary artery was also identified.
Additionally and importantly, the regression approach demonstrated in this study can be used to evaluate change in the quantity of CAC over time in an individual. For example, 21 participants had large increases between baseline and follow-up that indicated an increase in the quantity of CAC (baseline result to the left of the predicted range at follow-up in Fig 6). One man, aged 47 years, had a calcific area of 13.07 mm2 at baseline. At follow-up 3 years 2 months later, he had a calcific area of 100.29 mm2 and 108.89 mm2 in scan runs 1 and 2, respectively. Another man, aged 54 years, had a calcific area of 77.42 mm2 at baseline and 137.02 mm2 and 150.43 mm2 in scan runs 1 and 2, respectively, at follow-up 4 years 9 months later. One participant had a large decrease between baseline and follow-up that indicated a decrease in the detected quantity of CAC (baseline result to the right of the predicted range at follow-up in Fig 6). This man, aged 47 years, had a calcific area of 5.07 mm2 at baseline and a calcific area of 0 mm2 in both scan runs 1 and 2 at follow-up approximately 3 years 9 months later. Others (18) have reported apparent decreases in the quantity of CAC in specific individuals. To identify individuals with a change in the quantity of CAC, the 95% limits of agreement should be used in conjunction with careful inspection of the scan run findings by a radiologist.
The regression approach used to evaluate the agreement between two measures with nonuniform differences and demonstrated in this study can be used to compare the quantity of CAC in dual scan runs (acquisition of two sets of scans) in an individual performed minutes or years apart. By using the regression model results, the 95% limits of agreement can be calculated for any mean calcific area observed in the current study. This approach was appropriate for the observed distribution of quantity of CAC in participants in this asymptomatic study group who were not selected because they were at high risk for CAD. Future studies could be conducted to evaluate agreement in dual scan results in different study groups, separately in men and women, or among coronary arteries.
Scanner function and scanning protocols, however, may vary among institutions. Therefore, each institution should consider establishing their own 95% limits of agreement, which might vary depending on the scanning equipment, scanning and scoring protocols, or study group examined. The approach presented in the current study may be used to evaluate the agreement in dual scan run findings for alternate measures of CAC quantity, such as the Agatston score (28) and volumetric calcium scores (14). Similarly, results obtained with other image-processing software or scanning modalities such as spiral CT can be appropriately evaluated with this recently described method.
| FOOTNOTES |
|---|
Author contributions: Guarantors of integrity of entire study, L.F.B., P.A.P., P.F.S.; study concepts and design, L.F.B., P.F.S., P.A.P.; definition of intellectual content, L.F.B., P.F.S., P.A.P.; literature research, L.F.B.; clinical studies, P.F.S.; data acquisition, P.F.S.; data analysis, L.F.B., P.F.S., P.A.P.; statistical analysis, L.F.B.; manuscript preparation, editing, and review, L.F.B., P.F.S., P.A.P.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
H. Chung, R. L. McClelland, R. Katz, J. J. Carr, and M. J. Budoff Repeatability Limits for Measurement of Coronary Artery Calcified Plaque with Cardiac CT in the Multi-Ethnic Study of Atherosclerosis Am. J. Roentgenol., February 1, 2008; 190(2): W87 - W92. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Rutten, I. Isgum, and M. Prokop Coronary Calcification: Effect of Small Variation of Scan Starting Position on Agatston, Volume, and Mass Scores Radiology, December 1, 2007; 246(1): 90 - 98. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. E. Cassidy-Bushrow, L. F. Bielak, P. F. Sheedy II, S. T. Turner, I. J. Kullo, X. Lin, and P. A. Peyser Coronary Artery Calcification Progression Is Heritable Circulation, July 3, 2007; 116(1): 25 - 31. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. A. Kronmal, R. L. McClelland, R. Detrano, S. Shea, J. A. Lima, M. Cushman, D. E. Bild, and G. L. Burke Risk Factors for the Progression of Coronary Artery Calcification in Asymptomatic Subjects: Results From the Multi-Ethnic Study of Atherosclerosis (MESA) Circulation, May 29, 2007; 115(21): 2722 - 2730. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Messika-Zeitoun, L. F. Bielak, P. A. Peyser, P. F. Sheedy, S. T. Turner, V. T. Nkomo, J. F. Breen, J. Maalouf, C. Scott, A. J. Tajik, et al. Aortic Valve Calcification: Determinants and Progression in the Population Arterioscler. Thromb. Vasc. Biol., March 1, 2007; 27(3): 642 - 648. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. M. Iovannisci, E. J. Lammer, L. Steiner, S. Cheng, L. T. Mahoney, P. H. Davis, R. M. Lauer, and T. L. Burns Association Between A Leukotriene C4 Synthase Gene Promoter Polymorphism and Coronary Artery Calcium in Young Women: The Muscatine Study Arterioscler. Thromb. Vasc. Biol., February 1, 2007; 27(2): 394 - 399. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. J. Budoff, S. Achenbach, R. S. Blumenthal, J. J. Carr, J. G. Goldin, P. Greenland, A. D. Guerci, J. A.C. Lima, D. J. Rader, G. D. Rubin, et al. Assessment of Coronary Artery Disease by Cardiac Computed Tomography: A Scientific Statement From the American Heart Association Committee on Cardiovascular Imaging and Intervention, Council on Cardiovascular Radiology and Intervention, and Committee on Cardiac Imaging, Council on Clinical Cardiology Circulation, October 17, 2006; 114(16): 1761 - 1791. [Full Text] [PDF] |
||||
![]() |
F Cademartiri Is calcium the key for the assessment of progression/regression of coronary artery disease? Heart, September 1, 2006; 92(9): 1187 - 1188. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Schmermund, S. Achenbach, T. Budde, Y. Buziashvili, A. Forster, G. Friedrich, M. Henein, G. Kerkhoff, F. Knollmann, V. Kukharchuk, et al. Effect of Intensive Versus Standard Lipid-Lowering Treatment With Atorvastatin on the Progression of Calcified Coronary Atherosclerosis Over 12 Months: A Multicenter, Randomized, Double-Blind Trial Circulation, January 24, 2006; 113(3): 427 - 437. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. E. Clouse, J. Chen, H. M. Krumholz, M. E. Clouse, J. Chen, and H. M. Krumholz Noninvasive Screening for Coronary Artery Disease With Computed Tomography Is Useful Circulation, January 3, 2006; 113(1): 125 - 146. [Full Text] [PDF] |
||||
![]() |
A. B. Sevrukov, J. M. Bland, and G. T. Kondos Serial Electron Beam CT Measurements of Coronary Artery Calcium: Has Your Patient's Calcium Score Actually Changed? Am. J. Roentgenol., December 1, 2005; 185(6): 1546 - 1553. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. E. Cassidy, L. F. Bielak, Y. Zhou, P. F. Sheedy II, S. T. Turner, J. F. Breen, P. A. Araoz, I. J. Kullo, X. Lin, and P. A. Peyser Progression of Subclinical Coronary Atherosclerosis: Does Obesity Make a Difference? Circulation, April 19, 2005; 111(15): 1877 - 1882. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Schlosser, P. Hunold, A. Schmermund, H. Kuhl, K.-U. Waltering, J. F. Debatin, and J. Barkhausen Coronary Artery Calcium Score: Influence of Reconstruction Interval at 16-Detector Row CT with Retrospective Electrocardiographic Gating Radiology, November 1, 2004; 233(2): 586 - 589. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. Messika-Zeitoun, M.-C. Aubry, D. Detaint, L. F. Bielak, P. A. Peyser, P. F. Sheedy, S. T. Turner, J. F. Breen, C. Scott, A. J. Tajik, et al. Evaluation and Clinical Implications of Aortic Valve Calcification Measured by Electron-Beam Computed Tomography Circulation, July 20, 2004; 110(3): 356 - 362. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Abedin, Y. Tintut, and L. L. Demer Vascular Calcification: Mechanisms and Clinical Ramifications Arterioscler. Thromb. Vasc. Biol., July 1, 2004; 24(7): 1161 - 1170. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. E. Hokanson, T. MacKenzie, G. Kinney, J. K. Snell-Bergeon, D. Dabelea, J. Ehrlich, R. H. Eckel, and M. Rewers Evaluating Changes in Coronary Artery Calcium: An Analytic Method That Accounts for Interscan Variability Am. J. Roentgenol., May 1, 2004; 182(5): 1327 - 1332. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. J. Budoff Tracking Progression of Heart Disease with Cardiac Computed Tomography Journal of Cardiovascular Pharmacology and Therapeutics, April 1, 2004; 9(2): 75 - 82. [Abstract] [PDF] |
||||
![]() |
R. Vliegenthart, B. Song, A. Hofman, J. C. M. Witteman, and M. Oudkerk Coronary Calcification at Electron-Beam CT: Effect of Section Thickness on Calcium Scoring in Vitro and in Vivo Radiology, November 1, 2003; 229(2): 520 - 525. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. R. Van Hoe, K. G. De Meerleer, P. Ph. Leyman, and P. K. Vanhoenacker Coronary Artery Calcium Scoring Using ECG-Gated Multidetector CT: Effect of Individually Optimized Image-Reconstruction Windows on Image Quality and Measurement Reproducibility Am. J. Roentgenol., October 1, 2003; 181(4): 1093 - 1100. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. A. Rumberger and L. Kaufman A Rosetta Stone for Coronary Calcium Risk Stratification: Agatston, Volume, and Mass Scores in 11,490 Individuals Am. J. Roentgenol., September 1, 2003; 181(3): 743 - 748. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. S. Sonnad Describing Data: Statistical and Graphical Methods Radiology, December 1, 2002; 225(3): 622 - 628. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Achenbach, D. Ropers, K. Pohle, A. Leber, C. Thilo, A. Knez, T. Menendez, R. Maeffert, M. Kusus, M. Regenfus, et al. Influence of Lipid-Lowering Therapy on the Progression of Coronary Artery Calcification: A Prospective Evaluation Circulation, August 27, 2002; 106(9): 1077 - 1082. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. A. Peyser, L. F. Bielak, J. S. Chu, S. T. Turner, D. L. Ellsworth, E. Boerwinkle, and P. F. Sheedy II Heritability of Coronary Artery Calcium Quantity Measured by Electron Beam Computed Tomography in Asymptomatic Adults Circulation, July 16, 2002; 106(3): 304 - 308. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Pohle, R. Maffert, D. Ropers, W. Moshage, N. Stilianakis, W. G. Daniel, and S. Achenbach Progression of Aortic Valve Calcification: Association With Coronary Atherosclerosis and Cardiovascular Risk Factors Circulation, October 16, 2001; 104(16): 1927 - 1932. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. D. Hopper, D. C. Strollo, and D. T. Mauger Comparison of Electron-Beam and Ungated Helical CT in Detecting Coronary Arterial Calcification by Using a Working Heart Phantom and Artificial Coronary Arteries Radiology, February 1, 2002; 222(2): 474 - 482. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||