Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Published online before print May 16, 2007, 10.1148/radiol.2441060749
This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2441060749v1
244/1/213    most recent
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by U-King-Im, J. M.
Right arrow Articles by Gillard, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by U-King-Im, J. M.
Right arrow Articles by Gillard, J. H.
(Radiology 2007;244:213-222.)
© RSNA, 2007

Head and Neck Imaging

Internal Carotid Artery Stenosis: Accuracy of Subjective Visual Impression for Evaluation with Digital Subtraction Angiography and Contrast-enhanced MR Angiography1

Jean Marie U-King-Im, MRCS, Martin J. Graves, MSc, Justin J. Cross, FRCR, Nicholas J. Higgins, FRCR, Josephine Wat, FRCR, Rikin A. Trivedi, MRCS, Tjun Tang, MRCS, Simon P. S. Howarth, MRCS, Peter J. Kirkpatrick, FRCS, Nagui M. Antoun, FRCR, and Jonathan H. Gillard, FRCR

1 From the University Department of Radiology (J.M.U., M.J.G., J.J.C., N.J.H., J.W., T.T., S.P.S.H., N.M.A., J.H.G.) and Department of Neurosurgery (R.A.T., P.J.K.), Addenbrooke's Hospital, Box 219, Cambridge CB2 2QQ, England. Received April 29, 2006; revision requested June 27; revision received July 17; accepted August 23; final version accepted November 1. Supported by a grant from the National Health Service Research and Development Programme. Address correspondence to J.H.G. (e-mail: jhg21{at}cam.ac.uk).


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Purpose: To prospectively determine, for both digital subtraction angiography (DSA) and contrast material–enhanced magnetic resonance (MR) angiography, the accuracy of subjective visual impression (SVI) in the evaluation of internal carotid artery (ICA) stenosis, with objective caliper measurements serving as the reference standard.

Materials and Methods: Local ethics committee approval and written informed patient consent were obtained. A total of 142 symptomatic patients (41 women, 101 men; mean age, 70 years; age range, 44–89 years) suspected of having ICA stenosis on the basis of Doppler ultrasonographic findings underwent both DSA and contrast-enhanced MR angiography. With each modality, three independent neuroradiologists who were blinded to other test results first visually estimated and subsequently objectively measured stenoses. Diagnostic accuracy and percentage misclassification for correct categorization of 70%–99% stenosis were calculated for SVI, with objective measurements serving as the reference standard. Interobserver variability was determined with {kappa} statistics.

Results: After exclusion of arteries that were unsuitable for measurement, 180 vessels remained for analysis with DSA and 159 vessels remained for analysis with contrast-enhanced MR angiography. With respect to 70%–99% stenosis, SVI was associated with average misclassification of 8.9% for DSA (8.9%, 7.8%, and 10.0% for readers A, B, and C, respectively) and of 11.7% for contrast-enhanced MR angiography (11.3%, 8.8%, and 15.1% for readers A, B, and C, respectively). Negative predictive values were excellent (92.3%–100%). Interobserver variability was higher for SVI (DSA, {kappa} = 0.62–0.71; contrast-enhanced MR angiography, {kappa} = 0.57–0.69) than for objective measurements (DSA, {kappa} = 0.75–0.80; contrast-enhanced MR angiography, {kappa} = 0.66–0.72).

Conclusion: SVI alone is not recommended for evaluation of ICA stenosis with both DSA and contrast-enhanced MR angiography. SVI may be acceptable as an initial screening tool to exclude the presence of 70%–99% stenosis, but caliper measurements are warranted to confirm the presence of such stenosis.

© RSNA, 2007


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
The results of large randomized controlled trials, such as the North American Symptomatic Carotid Endarterectomy Trial (NASCET) and the European Carotid Surgery Trial (ECST), have clearly demonstrated the benefits of carotid endarterectomy over medical therapy in recently symptomatic patients with severe (70%–99%) internal carotid artery (ICA) stenosis, as measured with the NASCET method (1,2). More recently, carotid artery stent placement has emerged as an effective and less invasive alternative to carotid endarterectomy (3). For symptomatic patients, eligibility criteria for both the Carotid Revascularization Endarterectomy versus Stenting Trial, or CREST, and the Carotid Revascularization Using Endarterectomy or Stenting Systems, or CaRESS, trial included a NASCET stenosis of 50%–99% (4,5). Similar stenosis criteria apply for reimbursement eligibility by the Center for Medicare Services in symptomatic patients at high risk for surgery, although 50%–69% stenosis is only covered under the clinical trials policy (6).

Several measurement techniques, such as the NASCET, ECST, and common carotid methods, are used worldwide to assess the severity of stenosis (7). Regardless of the measurement method used, in the context of the original NASCET and ECST trials the degree of stenosis was calculated from objective and precise measurements of arterial diameter obtained with calipers. In the clinical setting, however, the use of subjective visual impression (SVI) of stenosis may be more widespread than is generally acknowledged (8–11). Although SVI data are rather scarce, a survey of U.S. teaching hospitals conducted by Chaturvedi et al (12) revealed that although the NASCET measurement method was unsurprisingly and overwhelmingly favored, 22.7% of centers acknowledged use of SVI alone or in combination with NASCET or ECST methods. Although use of SVI has been reported with conventional digital subtraction angiography (DSA) (13,14), less attention has been focused on noninvasive modalities, such as contrast material–enhanced magnetic resonance (MR) angiography, which is rapidly increasing in popularity (13–17). Thus, the purpose of our study was to prospectively determine, for both DSA and contrast-enhanced MR angiography, the accuracy of SVI in the evaluation of ICA stenosis, with objective caliper measurements serving as the reference standard.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Patients
Between August 2000 and January 2003, 167 consecutive symptomatic patients suspected of having ICA stenosis on the basis of screening Doppler ultrasonographic (US) images that depicted at least 50% stenosis in one or more arteries were initially prospectively recruited to undergo both DSA and contrast-enhanced MR angiography. Patients with contraindications to MR imaging, such as claustrophobia or metallic implants, were excluded. This study and data release were approved by the local ethics committee, and all patients gave informed written consent. Twenty-five patients did not undergo complete DSA or contrast-enhanced MR angiography for various reasons; thus, 142 patients (41 women, 101 men; mean age, 70 years; age range, 44–89 years) underwent both DSA and contrast-enhanced MR angiography.

Diagnostic Tests
One of four attending neuroradiologists (J.H.G., J.J.C., N.J.H., or N.M.A.; 2, 2, 10, and 18 years of experience, respectively, with DSA at the start of the study) performed DSA with bilateral selective common carotid artery catheterization by using a single-plane digital angiographic unit (Angioskop; Siemens, Erlangen, Germany). Images of each bifurcation were acquired in four projections (anteroposterior, lateral, and bilateral 45° oblique) with a 33-cm field of view, a 1024 x 1024 matrix, and a 0.32 x 0.32-mm spatial resolution and were recorded on film hard copy.

Contrast-enhanced MR angiography was performed with a 1.5-T machine (Signa CV/i; GE Medical Systems, Milwaukee, Wis) with use of a transmit-receive neurovascular coil (Medrad, Indianola, Pa). A bolus-timed three-dimensional breath-hold fast spoiled gradient-echo acquisition (repetition time msec/fractional echo time msec, 5.3/1.6; 44° flip angle; 62.5-kHz bandwidth; 23 x 16-cm field of view; 0.8-mm section thickness; 256 x 256 matrix; 0.5 signal acquired) with an elliptic centric phase-encode ordering scheme was performed after administration of 30 mL of gadolinium-based contrast material (gadopentetate dimeglumine, Magnevist; Schering, Berlin, Germany) at a flow rate of 2 mL/sec with a power injector. Zero-filling interpolation was used and resulted in a final in-plane resolution of 0.60 x 0.80 mm. Each contrast-enhanced MR angiographic data set was transferred to a dedicated workstation (Advantage 4.0; GE Medical Systems) to be processed by an experienced technologist with use of a maximum intensity projection algorithm. After exclusion of the vertebral arteries, each carotid bifurcation was recorded on film hard copy in the same four projections used for DSA.

Image Analysis
Three independent readers (J.J.C., N.J.H., and N.M.A., with 3, 7, and 19 years of experience, respectively, as attending neuroradiologists at the start of image analysis in 2003) who were blinded to clinical history and results of other diagnostic tests reviewed all DSA and contrast-enhanced MR angiographic studies. However, the readers were aware that screening Doppler US had depicted at least 50% stenosis in at least one ICA. Patient identifiers on film hard copies were masked. Image analysis was performed in batches after patient recruitment was finished. The DSA images were randomized and evaluated over a 4-month period. The contrast-enhanced MR angiograms were then subsequently evaluated over the following 4 months in a different randomized order. Each left and right ICA was evaluated as an independent unit.

For each study, each neuroradiologist first chose the projection that demonstrated the most severe stenosis and then assigned an SVI by estimating the percentage of stenosis; an independent observer (J.M.U.) who was present during all readings immediately recorded this finding in a database. Thereafter, each neuroradiologist obtained two objective measurements with a digital Vernier caliper calibrated to the nearest 0.1 mm: Measurement A was the luminal diameter at the site of maximal narrowing, and measurement B was the luminal diameter of the normal distal ICA. In strict accordance with NASCET guidelines, care was taken to ensure that measurement B was obtained in the distal ICA, which was well beyond the bulb and where the artery walls became parallel (Fig 1a) (18–21). SVI and caliper measurements were obtained during the same session to eliminate the effects of intraobserver variability that otherwise may have occurred if reading had been performed during different sessions (22). The neuroradiologist was not permitted to revise his SVI reading after objective measurements were obtained. The independent observer recorded only the actual diameter measurements in the database at the time of image interpretation. The percentage of stenosis was calculated at a later stage by the independent observer in the absence of the neuroradiologist according to the formula S = [(B – A)/B] x 100, where A refers to measurement A, B refers to measurement B, and S refers to NASCET stenosis.


Figure 1A
View larger version (68K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1a: (a) Left lateral DSA image of the ICA. Residual luminal diameter is measured at the position of maximal stenosis (A). The correct measurement of the denominator needs to be obtained well distal to the bulb, where the artery walls have become parallel (B2). If the denominator measurement is obtained more proximally (B1), this tends to lead to overestimation of stenosis. (b) DSA image obtained in the 45° oblique projection shows near occlusion of a left ICA. The caliber of the distal ICA (arrows) is markedly reduced beyond a critical proximal stenosis; objective NASCET measurements are not meaningful in such studies.

 

Figure 1B
View larger version (68K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1b: (a) Left lateral DSA image of the ICA. Residual luminal diameter is measured at the position of maximal stenosis (A). The correct measurement of the denominator needs to be obtained well distal to the bulb, where the artery walls have become parallel (B2). If the denominator measurement is obtained more proximally (B1), this tends to lead to overestimation of stenosis. (b) DSA image obtained in the 45° oblique projection shows near occlusion of a left ICA. The caliber of the distal ICA (arrows) is markedly reduced beyond a critical proximal stenosis; objective NASCET measurements are not meaningful in such studies.

 
Data Analysis
The purpose of this study was not to compare DSA with contrast-enhanced MR angiography but rather to compare SVI with objective caliper measurements obtained independently on DSA images and contrast-enhanced MR angiograms. Thus, for each SVI interpretation performed by a neuroradiologist using a given modality, the reference standard was the percentage of stenosis, as determined with objective caliper measurements obtained by the same neuroradiologist using the same modality.

There are several scenarios, including normal arteries, occluded arteries, and nearly occluded arteries, in which objective caliper measurements cannot be obtained. Near occlusion of the ICA refers to the appearance of partial luminal diameter decrease (lumen diameter similar to or less than that of the ipsilateral external carotid artery and less than that of the contralateral ICA) of the nondiseased distal ICA beyond a proximal critical carotid bulb stenosis (Fig 1b) (23,24). Because of the apparent decrease in ICA diameter as a result of slow contrast medium flow, caliper measurements tend to lead to underestimation of the degree of stenosis. Thus, in clinical practice, these studies are arbitrarily classified as showing near occlusions without measurements (23). Moreover, for contrast-enhanced MR angiographic studies only, an ICA can show a flow void or signal gap, both of which have been shown to be equivalent to 70%–99% stenosis (17,25). For such studies, caliper measurements are again impossible to obtain, as there is no residual lumen. After exclusion of such studies from data analysis, stenosis was classified as 0%–49% (mild), 50%–69% (moderate), or 70%–99% (severe).

Statistical Analysis
All data were analyzed with SPSS statistical software for Windows (version 10.1; SPSS, Chicago, Ill). Overall diagnostic misclassification was defined as the percentage of studies in which SVI and objective caliper measurements disagreed as to the correct assessment of stenosis into the 0%–49%, 50%–69%, or 70%–99% category. Sensitivity, specificity, and predictive values with 95% confidence intervals were calculated for the accuracy of SVI in the correct classification of 70%–99% stenosis and 50%–99% stenosis. These two groups (ie, 70%–99% stenosis and 50%–99% stenosis) represent generally accepted stenosis criteria for intervention in the form of carotid endarterectomy and carotid artery stent placement, respectively (4,24,26). Misclassification for correct categorization into the 70%–99% or 50%–99% group was defined as the percentage of studies in which there was disagreement as to the presence or absence of these stenosis groups. Moreover, among all cases of misclassification for categorization as 70%–99% stenosis, instances in which SVI and caliper readings were within 4% of each other were identified. We arbitrarily chose 4% because typical measurement errors for inter- and intraobserver variability for the reference standard (DSA) have been shown to be of the order of ±5% (9,27).

Agreement between the two readings was assessed with the Bland-Altman method for continuous agreement and Cohen {kappa} statistics for categorical agreement regarding 0%–49%, 50%–69%, and 70%–99% stenosis (28). {kappa} values of 0.80–1.00 were taken to represent good agreement; {kappa} values of 0.60–0.79, moderate agreement; and {kappa} values of 0.00–0.59, poor agreement. Finally, interobserver agreement was determined for both contrast-enhanced MR angiography and DSA with {kappa} statistics and Bland-Altman methods. The Fisher exact test and {chi}2 tests were used to determine statistical significance, and P ≤ .05 indicated a statistically significant difference.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Study Population
The 142 patients who underwent both DSA and contrast-enhanced MR angiography were initially included, yielding 284 arteries for initial analysis. An intention-to-image approach was adopted, and no study was excluded on the basis of poor technical quality. However, after exclusion of 37 occluded arteries in 33 patients, 58 normal arteries in 51 patients, and nine nearly occluded arteries in nine patients, 180 arteries in 124 patients remained for analysis of DSA findings. Moreover, for contrast-enhanced MR angiography only, 21 additional studies in 20 patients with flow voids were excluded, yielding 159 arteries in 118 patients for analysis. The patient flow diagram constructed according to the STARD (Standard for Reporting of Diagnostic Accuracy) criteria is shown in Figure 2 (29).


Figure 2
View larger version (15K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2: Patient flow chart constructed according to STARD (Standards for Reporting of Diagnostic Accuracy) recommendations. CEMRA = contrast-enhanced MR angiography, MRA = MR angiography.

 
Agreement between SVI and Caliper Measurements
Categorical agreement between SVI and caliper measurements was moderate for DSA, with {kappa} values of 0.78, 0.74, and 0.71 for readers A, B, and C, respectively. Categorical agreement for contrast-enhanced MR angiography was also moderate, although it was not as good as that between SVI and caliper measurements for DSA, with corresponding {kappa} values of 0.71, 0.66, and 0.62. These differences between DSA and contrast-enhanced MR angiography were significant for readers A, B, and C (P = .03, P = .03, and P = .02, respectively).

Bland-Altman plots were used to assess differences between SVI and caliper measurements at DSA and contrast-enhanced MR angiography for each reader (Fig 3) and showed different patterns that ranged from underestimation to overestimation with SVI among the different readers, with mean biases (ie, the mean difference between the two readings) that ranged from –4.9% (reader A using DSA) to 3.7% (reader B using DSA). For contrast-enhanced MR angiography, mean biases ranged from –1.3% to 2.5%. The limits of agreement, within which 95% of the differences between the two measurement methods (SVI vs caliper) were expected to lie, ranged from ±14.1% to ±22.7% for DSA and from ±14.7% to ±20.9% for contrast-enhanced MR angiography, with no definite trend that emerged.


Figure 3A
View larger version (14K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3a: Bland-Altman plots for agreement between SVI and objective caliper measurements for (a) DSA and (b) contrast-enhanced MR angiography (CEMRA) on a per-reader basis. Difference in the percentage of stenosis between SVI and caliper measurements is plotted against the mean value of the two readings. Solid line represents mean bias between SVI and caliper measurements. Dotted lines represent upper and lower limits of agreement, within which 95% of the differences between SVI and caliper measurements are expected to lie.

 

Figure 3B
View larger version (14K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3b: Bland-Altman plots for agreement between SVI and objective caliper measurements for (a) DSA and (b) contrast-enhanced MR angiography (CEMRA) on a per-reader basis. Difference in the percentage of stenosis between SVI and caliper measurements is plotted against the mean value of the two readings. Solid line represents mean bias between SVI and caliper measurements. Dotted lines represent upper and lower limits of agreement, within which 95% of the differences between SVI and caliper measurements are expected to lie.

 
Overall Diagnostic Misclassification
For readers A, B, and C, respectively, overall diagnostic misclassifications were 14.4%, 17.2%, and 18.9% for DSA and 18.9%, 22.0%, and 23.9% for contrast-enhanced MR angiography (Table 1). Although there was a tendency for misclassification to be higher for contrast-enhanced MR angiography than for DSA for each reader, this difference was not significant for any reader (P = .30, P = .27, and P = .24 for readers A, B, and C, respectively; Fisher exact test).


View this table:
[in this window]
[in a new window]

  		Table 1. Summary of Diagnostic Performance of SVI with Caliper Measurements as the Reference Standard for DSA and Contrast-enhanced MR Angiography on a Per-Reader Basis

 
Most cases of misclassification for either modality stemmed from overestimation with SVI, although there were some important interobserver differences (Tables 1, 2). The proportion of cases of overestimation that accounted for overall misclassification was high for readers B (DSA, 93.5%; contrast-enhanced MR angiography, 91.4%) and C (DSA, 94.1%; contrast-enhanced MR angiography, 73.7%). However, the SVI assigned by reader A was more balanced, with overestimation accounting for only 53.8% of errors for DSA and 60.0% of errors for contrast-enhanced MR angiography.


View this table:
[in this window]
[in a new window]

  		Table 2. Agreement between SVI and Objective Caliper Measurements with DSA and Contrast-enhanced MR Angiography

 
Misclassification of Categorization into 70%–99% and 50%–99% Stenosis Groups
With regard to the presence or absence of 70%–99% stenosis (Table 1), there was a spectrum of values for sensitivity (87.3%–96.7%) and specificity (83.1%–100%) that ranged from moderate to excellent for each reader. However, the consistently excellent negative predictive values ranged from 92.3% to 100%.

For readers A, B, and C, respectively, misclassifications in the categorization of 70%–99% stenosis were 8.9%, 7.8%, and 10.0% for DSA and 11.3%, 8.8%, and 15.1% for contrast-enhanced MR angiography (Table 1). Again, there was a tendency for these misclassification rates to be higher for contrast-enhanced MR angiography than for DSA within each reader; this difference was not significant for readers A or B (P = .11 and P = .14, respectively) but was significant for reader C (P = .049). Individual review of such cases of misclassification showed that the mean number of studies where SVI and caliper measurements were within 4% of each other was three (range, two to five studies) for each data set.

Regarding the sensitivity, specificity, and predictive values for correct categorization of 50%–99% stenosis with SVI (Table 1), negative predictive values were excellent and ranged from 90.0% to 98.1%. There was a tendency for misclassification with SVI to be slightly higher for contrast-enhanced MR angiography (7.5%–13.2%) than for DSA (6.1%–9.4%), but this difference was not significant (P = .12, P = .09, and P = .11 for readers A, B, and C, respectively).

Interobserver Agreement
For both DSA and contrast-enhanced MR angiography (Table 3), interobserver variability was greater for SVI measurements than for caliper measurements. Thus, for DSA, {kappa} values ranged from 0.75 to 0.80 for caliper measurements and from 0.62 to 0.71 for SVI measurements. Similarly, for contrast-enhanced MR angiography, {kappa} values were better for caliper measurements (0.66–0.72) than for SVI measurements (0.57–0.69). Consistent with this pattern, Bland-Altman analysis revealed undeniably wider limits of agreement for SVI (range, ±19.1 to ±25.6%) than for objective caliper measurements (range, ±13.7% to ±20.8%), reflecting the poorer interobserver variability of SVI.


View this table:
[in this window]
[in a new window]

  		Table 3. Interobserver Agreement of SVI and Objective Caliper Measurements for DSA and Contrast-enhanced MR Angiography

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
ADVANCES IN KNOWLEDGE
 References
 
There are several reasons why radiologists and clinicians tend to use SVI rather than obtain objective measurements. Miller (30) argued that as radiologists, we are often taught to train our eyes to identify and characterize abnormalities without the need for objective quantification. Frequently, a ruler or caliper may not be available at the time of reporting. Also, the more confident the radiologist is, the more he or she may believe that objective quantification will add little to the careful qualitative evaluation based on his or her unique expertise acquired during many years of practice. Moreover, for ICA stenosis, it is arguable whether SVI can enable one to integrate information from different viewing angles into one result and thereby eliminate some variability due to disagreement over which projection to assess.

Data that are likely to be most relevant to clinical practice probably relate to the presence or absence of 70%–99% stenosis for surgery and of 50%–99% stenosis for stent placement.

For 70%–99% stenosis, misclassification with SVI ranged from 8.9% to 10.0% for DSA and from 11.3% to 15.1% for contrast-enhanced MR angiography in our study. Similarly, for 50%–99% stenosis, misclassification with SVI ranged from 6.1% to 9.4% for DSA and from 7.5% to 13.2% for contrast-enhanced MR angiography. SVI had consistently higher interobserver variability when compared with caliper measurements for both modalities. This is not surprising, as one could intuitively expect precisely defined measurements to be less dependent on the observer and therefore more readily generalizable. Overall, however, interobserver variability in our study was consistent with previously reported data (31).

In our opinion, the misclassification and higher interobserver variability were far from negligible and do not support the use of SVI alone in the evaluation of ICA stenosis. It is noteworthy, however, that as far as exclusion of the presence of 70%–99% stenosis or 50%–99% stenosis was concerned, SVI performed well for all three readers with both DSA and contrast-enhanced MR angiography, with consistently excellent negative predictive values. Thus, SVI may be acceptable, at least when performed by experienced readers, as an initial screening tool. If SVI reveals a stenosis to be less than 50% or 70%, caliper measurements are not necessary. Objective measurements are warranted, however, to confirm the presence of either 50%–99% or 70%–99% stenosis.

Our data largely corroborate the findings of Pelz et al (13) who examined the use of SVI in the evaluation of angiograms of 125 patients in the NASCET trial. Three independent groups of reviewers who consisted of neuroradiologists, neurologists, neurosurgeons, and general radiologists read the angiograms. Use of SVI resulted in false-positive rates (for assessing stenosis of less than 70% as severe) that ranged from 5% among neuroradiologists to 16% among general radiologists. Similarly, the false-negative rates (for assessing stenosis of 70% or more as less than severe) ranged from 12% to 22%. These findings suggests that SVI may be dependent on the reader's experience and training.

In contrast, in a study by Schmittling et al (14) in which SVIs assigned by vascular surgeons for 200 ICA angiograms were compared with objective measurements obtained with digital caliper computer software, SVI performed substantially worse than it did in our study. Misclassification was high, ranging from 40% to 56% for the 16%–49% stenosis group, from 30% to 44% for the 50%–79% stenosis group, and from 27% to 51% for the 80%–99% stenosis group. In the Schmittling et al study, all errors were due to overestimation (ie, SVI never resulted in underestimation of ICA stenosis).

Both Pelz et al (13) and Schmittling et al (14) investigated conventional DSA only. Given the growing acceptance of noninvasive modalities as alternatives to DSA, it is interesting that use of SVI with contrast-enhanced MR angiography generally performed worse than did use of SVI with DSA. There was a tendency for higher diagnostic and clinically relevant misclassification with SVI and contrast-enhanced MR angiograms compared with that with SVI and DSA images, although these differences were not always significant. However, a significant finding was that categorical agreement between SVI and objective measurements was worse for contrast-enhanced MR angiography than for DSA. Moreover, for both SVI and objective measurements, interobserver variability was also consistently and significantly higher for contrast-enhanced MR angiography than for DSA. The explanation for this is probably spatial resolution differences. Although we achieved submillimeter spatial resolution (0.60 x 0.80 mm) with contrast-enhanced MR angiography, this was still two or three times less than that achieved with conventional DSA (0.32 x 0.32 mm). With technical advances in MR angiographic techniques, such as the use of parallel imaging or higher field strength, such differences between DSA and MR angiography will probably decrease (32,33).

We used film hard copies because this was the current practice at our institution at the time of reporting. We acknowledge this as a limitation of our study, given the increasing use of reporting at workstations; however, we suggest that there are no convincing reasons why our conclusions regarding SVI would not also apply to soft-copy images. Thus, in our opinion, the use of electronic calipers or measuring tools that are routinely available on picture archiving and communications systems is recommended.

Another study limitation is that we used an absolute threshold of 70% to indicate severe stenosis on the basis of NASCET and ECST findings (1,2). Thus, a stenosis of 70% measured with SVI that was objectively measured and found to be 67% would be considered a misclassification, as it is likely that such small differences are not clinically important. This must be placed into context, as it is clear from our data that readers tend to assign SVIs of ICA stenosis to at least the nearest 5%. Moreover, even objective measurements of stenosis are not true reference standards, as they are subject to inter- and intraobserver variability (27). We therefore acknowledge that our described misclassifications could result in an overestimation of the number of times SVI would have made an important difference in patient care. Nevertheless, we manually counted all instances in which SVI readings were within 4% of caliper measurements but were still categorized as misclassifications because they straddled the 70% threshold. Such studies were few and ranged from a minimum of two to a maximum of five studies for each data set, implying that these misclassifications were unlikely to substantially affect our overall conclusions.

In conclusion, SVI of ICA stenosis is associated with significantly higher interobserver variability compared with objective caliper measurements. Even when performed by experienced neuroradiologists, SVI alone is not recommended for evaluation of ICA stenosis with either DSA or contrast-enhanced MR angiography. SVI may be acceptable as an initial screening tool to exclude the presence of 50%–99% or 70%–99% stenosis, but confirmatory caliper measurements are warranted to confirm the presence of such categories of stenosis.


   ADVANCES IN KNOWLEDGE
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
References
 


   FOOTNOTES
 

Abbreviations: DSA = digital subtraction angiography • ECST = European Carotid Surgery Trial • ICA = internal carotid artery • NASCET = North American Symptomatic Carotid Endarterectomy Trial • SVI = subjective visual impression

Authors stated no financial relationship to disclose.

Author contributions:Guarantor of integrity of entire study, J.H.G.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, J.M.U., T.T.; clinical studies, J.M.U., M.J.G., N.J.H., J.W., R.A.T., J.H.G.; statistical analysis, J.M.U.; and manuscript editing, J.M.U., M.J.G., J.J.C., R.A.T., T.T., S.P.S.H., J.H.G.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 

  1. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998;351(9113):1379–1387.
  2. Barnett HJ, Taylor DW, Eliasziw M; and the North American Symptomatic Carotid Endarterectomy Trial Collaborators. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. N Engl J Med 1998;339(20):1415–1425.[Abstract/Free Full Text]
  3. Roubin GS, Iyer S, Halkin A, Vitek J, Brennan C. Realizing the potential of carotid artery stenting: proposed paradigms for patient selection and procedural technique. Circulation 2006;113(16):2021–2030.[Abstract/Free Full Text]
  4. Hobson RW 2nd, Howard VJ, Brott TG, Howard G, Roubin GS, Ferguson RD. Organizing the Carotid Revascularization Endarterectomy versus Stenting Trial (CREST): National Institutes of Health, Health Care Financing Administration, and industry funding. Curr Control Trials Cardiovasc Med 2001;2(4):160–164.[CrossRef][Medline]
  5. CaRESS Steering Committee. Carotid Revascularization Using Endarterectomy or Stenting Systems (CaRESS) Phase I Clinical Trial: 1-year results. J Vasc Surg 2005;42(2):213–219.
  6. Meskan T. Carotid artery stenting: anatomy of a Medicare coverage decision. J Cardiovasc Manag 2005;16(2):7–11.[Medline]
  7. Rothwell PM, Gibson RJ, Slattery J, Sellar RJ, Warlow CP; for the European Carotid Surgery Trialists' Collaborative Group. Equivalence of measurements of carotid stenosis: a comparison of three methods on 1001 angiograms. Stroke 1994;25(12):2435–2439.[Abstract]
  8. Chang YJ, Golby AJ, Albers GW. Detection of carotid stenosis: from NASCET results to clinical practice. Stroke 1995;26(8):1325–1328.[Abstract/Free Full Text]
  9. Young GR, Humphrey PR, Nixon TE, Smith ET. Variability in measurement of extracranial internal carotid artery stenosis as displayed by both digital subtraction and magnetic resonance angiography: an assessment of three caliper techniques and visual impression of stenosis. Stroke 1996;27(3):467–473.[Abstract/Free Full Text]
  10. Vanninen R, Manninen H, Koivisto K, Tulla H, Partanen K, Puranen M. Carotid stenosis by digital subtraction angiography: reproducibility of the European Carotid Surgery Trial and the North American Symptomatic Carotid Endarterectomy Trial measurement methods and visual interpretation. AJNR Am J Neuroradiol 1994;15(9):1635–1641.[Abstract]
  11. Nonent M, Serfaty JM, Nighoghossian N, et al. Concordance rate differences of 3 noninvasive imaging techniques to measure carotid stenosis in clinical routine practice: results of the CARMEDAS multicenter study. Stroke 2004;35(3):682–686.[Abstract/Free Full Text]
  12. Chaturvedi S, Policherla PN, Femino L. Cerebral angiography practices at US teaching hospitals: implications for carotid endarterectomy. Stroke 1997;28(10):1895–1897.[Abstract/Free Full Text]
  13. Pelz DM, Fox AJ, Eliasziw M, Barnett HJ. Stenosis of the carotid bifurcation: subjective assessment compared with strict measurement guidelines. Can Assoc Radiol J 1993;44(4):247–252.[Medline]
  14. Schmittling ZC, McLafferty RB, Danetz JS, Hussain SM, Ramsey DE, Hodgson KJ. The inaccuracy of simple visual interpretation for measurement of carotid stenosis by arteriography. J Vasc Surg 2005;42(1):62–66.[CrossRef][Medline]
  15. Nederkoorn PJ, Elgersma OE, van der Graaf Y, Eikelboom BC, Kappelle LJ, Mali WP. Carotid artery stenosis: accuracy of contrast-enhanced MR angiography for diagnosis. Radiology 2003;228(3):677–682.[Abstract/Free Full Text]
  16. Alvarez-Linera J, Benito-Leon J, Escribano J, Campollo J, Gesto R. Prospective evaluation of carotid artery stenosis: elliptic centric contrast-enhanced MR angiography and spiral CT angiography compared with digital subtraction angiography. AJNR Am J Neuroradiol 2003;24(5):1012–1019.[Abstract/Free Full Text]
  17. U-King-Im JM, Trivedi RA, Graves MJ, et al. Contrast-enhanced MR angiography for carotid disease: diagnostic and potential clinical impact. Neurology 2004;62(8):1282–1290.[Abstract/Free Full Text]
  18. Barnett HJ, Warlow CP. Carotid endarterectomy and the measurement of stenosis. Stroke 1993;24(9):1281–1284.[Free Full Text]
  19. Eliasziw M, Fox AJ, Sharpe BL, Barnett HJ. Carotid artery stenosis: external validity of the North American Symptomatic Carotid Endarterectomy Trial measurement method. Radiology 1997;204(1):229–233.[Abstract/Free Full Text]
  20. Eliasziw M, Smith RF, Singh N, Holdsworth DW, Fox AJ, Barnett HJ; for the North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Further comments on the measurement of carotid stenosis from angiograms. Stroke 1994;25(12):2445–2449.[Abstract]
  21. Rothwell PM, Slattery J, Warlow CP. Measurement of carotid stenosis. Lancet 1994;344(8924):749–750.[CrossRef][Medline]
  22. Gagne PJ, Matchett J, MacFarland D, et al. Can the NASCET technique for measuring carotid stenosis be reliably applied outside the trial? J Vasc Surg 1996;24(3):449–455.[CrossRef][Medline]
  23. Fox AJ, Eliasziw M, Rothwell PM, Schmidt MH, Warlow CP, Barnett HJ. Identification, prognosis, and management of patients with carotid artery near occlusion. AJNR Am J Neuroradiol 2005;26(8):2086–2094.[Abstract/Free Full Text]
  24. Rothwell PM, Eliasziw M, Gutnikov SA, et al. Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis. Lancet 2003;361(9352):107–116.[CrossRef][Medline]
  25. Nederkoorn PJ, van der Graaf Y, Eikelboom BC, van der Lugt A, Bartels LW, Mali WP. Time-of-flight MR angiography of carotid artery stenosis: does a flow void represent severe stenosis? AJNR Am J Neuroradiol 2002;23(10):1779–1784.[Abstract/Free Full Text]
  26. CARESS Steering Committee. Carotid revascularization using endarterectomy or stenting systems (CARESS): phase I clinical trial. J Endovasc Ther 2003;10(6):1021–1030.
  27. Heiserman JE. Measurement error of percent diameter carotid stenosis determined by conventional angiography: implications for noninvasive evaluation. AJNR Am J Neuroradiol 2005;26(8):2102–2107.[Abstract/Free Full Text]
  28. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1(8476):307–310.[CrossRef][Medline]
  29. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Radiology 2003;226(1):24–28.[Abstract/Free Full Text]
  30. Miller GM. A radiologist with a ruler. AJNR Am J Neuroradiol 2003;24(4):556.[Free Full Text]
  31. Vanninen RL, Manninen HI, Partanen PK, Tulla H, Vainio PA. How should we estimate carotid stenosis using magnetic resonance angiography? Neuroradiology 1996;38(4):299–305.[Medline]
  32. Hu HH, Madhuranthakam AJ, Kruger DG, Glockner JF, Riederer SJ. Combination of 2D sensitivity encoding and 2D partial fourier techniques for improved acceleration in 3D contrast-enhanced MR angiography. Magn Reson Med 2006;55(1):16–22.[CrossRef][Medline]
  33. Willinek WA, Gieseke J, von Falkenhausen M, et al. Sensitivity encoding (SENSE) for high spatial resolution time-of-flight MR angiography of the intracranial arteries at 3.0 T. Rofo 2004;176(1):21–26.[Medline]



This article has been cited by other articles:


Home page
 Am. J. Neuroradiol.Home page
F. Runck, R.P. Steiner, W.A. Bautz, and M.M. Lell
MR Imaging: Influence of Imaging Technique and Postprocessing on Measurement of Internal Carotid Artery Stenosis
AJNR Am. J. Neuroradiol., October 1, 2008; 29(9): 1736 - 1742.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2441060749v1
244/1/213    most recent
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by U-King-Im, J. M.
Right arrow Articles by Gillard, J. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by U-King-Im, J. M.
Right arrow Articles by Gillard, J. H.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE