|
|
||||||||
Vascular and Interventional Radiology |
1 From the Departments of Epidemiology and Biostatistics (K.V., M.G.M.H.) and Radiology (M.G.M.H.), Erasmus Medical Center Rotterdam, Rm EE21-40a, Dr Molewaterplein 50, 3015 GE Rotterdam, the Netherlands, and the Department of Health Policy and Management, Harvard School of Public Health, Boston, Mass (M.G.M.H.). Received February 17, 1999; revision requested May 3; final revision received September 29; accepted October 25. Supported in part by the Persoongerichte Impuls voor Onderzoeksgroepen met Nieuwe Ideëen voor Excellente Research (PIONIER) award from the Netherlands Organization for Scientific Research. Address correspondence to M.G.M.H. (e-mail: hunink@epib.fgg.eur.nl).
| ABSTRACT |
|---|
|
|
|---|
MATERIALS AND METHODS: Studies published between January 1984 and November 1998 were included if (a) gadolinium-enhanced MR angiography and/or color-guided duplex US were performed for evaluation of arterial stenoses and occlusions in the work-up for peripheral arterial disease of the lower extremities, (b) conventional angiography was the reference standard, and (c) absolute numbers of true-positive, false-negative, true-negative, and false-positive results were available or derivable.
RESULTS: With a random effects model, pooled sensitivity for MR angiography (97.5% [95% CI: 95.7%, 99.3%]) was higher than that for duplex US (87.6% [95% CI: 84.4%, 90.8%]). Pooled specificities were similar: 96.2% (95% CI: 94.4%, 97.9%) for MR angiography and 94.7% (95% CI: 93.2%, 96.2%) for duplex US. Summary receiver operating characteristic analysis demonstrated better discriminatory power for MR angiography than for duplex US. Regression coefficients for MR angiography versus US were 1.67 (95% CI: -0.23, 3.56) with adjustment for covariates, 2.11 (95% CI: 0.12, 4.09) without such adjustment, and 1.73 (95% CI: 0.44, 3.02) with a random effects model.
CONCLUSION: Gadolinium-enhanced MR angiography has better discriminatory power than does color-guided duplex US and is a highly sensitive and specific method, as compared with conventional angiography, for the work-up for peripheral arterial disease.
Index terms: Arteries, extremities Arteries, stenosis or obstruction, 92.72 Arteries, MR, 92.12942, 92.12943 Arteries, US, 92.12981, 92.12983 Magnetic resonance (MR), comparative studies, 92.12942, 92.12943 Ultrasound (US), comparative studies, 92.12981, 92.12983
| INTRODUCTION |
|---|
|
|
|---|
Duplex ultrasonography (US) has been shown to be a reliable noninvasive modality with fairly good sensitivity and specificity (3). The addition of color flow imaging to help guide duplex scanning improves the diagnostic performance (4). Duplex US is, however, operator dependent and labor intensive and does not provide a road map equivalent to that obtained with angiography. Gadolinium-enhanced magnetic resonance (MR) angiography is a relatively new minimally invasive imaging method used for the work-up of peripheral arterial disease. Intravenous administration of a gadolinium chelate is considered to be safe, and MR angiography provides high-quality three-dimensional images of the vascular system in a short time with high sensitivity and specificity (511). Disadvantages of MR angiography are the expense and the small number of cases in which the procedure is unsuccessful due to susceptibility artifacts or because the patient has claustrophobia.
A number of articles have recently been published on the topic of gadolinium-enhanced MR angiography, each a report of the experience from a single center with a limited number of patients. Before supporting MR angiography as a substitute for conventional (x-ray) angiography, the overall combined evidence should demonstrate whether MR angiography is a highly sensitive and specific modality. Furthermore, we are aware of no reports in which gadolinium-enhanced MR angiography was compared with color-guided duplex US. If a choice must be made between the imaging modalities, it should be largely dependent on the reported diagnostic accuracies of both methods, taking into account differences in applied positivity criteria and patient characteristics. A powerful tool for this kind of analysis is summary receiver operating characteristic (ROC) analysis (1214).
The purpose of this study was to summarize and compare the diagnostic performance of gadolinium-enhanced MR angiography and color-guided duplex US for the evaluation of arterial stenoses and occlusions in the work-up of peripheral arterial disease of the lower extremities and to compare both methods with conventional angiography.
| MATERIALS AND METHODS |
|---|
|
|
|---|
Articles (511,1539) were included in the analysis if they met the following criteria: (a) Gadolinium-enhanced MR angiography, color-guided duplex US, or both, were performed to demonstrate stenoses and occlusions of the arteries in the lower extremities; (b) results of conventional angiography were used as the reference standard; and (c) the absolute numbers of true-positive (TP), false-negative (FN), true-negative (TN), and false-positive (FP) results were available or could be derived for a defined cutoff criterion for angiographyusually a reduction in arterial diameter of more than 50%. In the case of multiple published reports by the same author(s) over a brief time, we tried to contact the author(s) to determine whether the patient populations overlapped. Use of the same patient population more than once in our analysis could bias the results. Among the articles (6,23,28,3941) in which data were reported for the same patient population, we included those (6,23,39) with a research question most relevant to our meta-analysis. Moreover, we tried to contact the authors of articles that reported a measure of agreement (eg,
statistic) between the comparison modality and angiography rather than the absolute numbers of TP, FN, TN, and FP results. One author (26) gave us the relevant data, but two (5,42) neglected to reply despite repeated requests (total of three requests to each).
Each author independently extracted the data from all articles by using a standardized spreadsheet. The authors were not blinded with regard to identifying information of the individual manuscripts because this has been shown to be unnecessary (43). Extracted data included variables related to study design, patient characteristics, diagnostic imaging protocol, and absolute numbers of TP, FN, TN, and FP results (Tables 14). The absolute numbers were most often available for arterial segments rather than for limbs or patients. If the results were tabulated for different readers, then the results of the first reader were used. If more than one examination technique was presented in the same article, we used the technique advised by the authors. US results for iliac segments were excluded if stenoses were indirectly determined on the basis of Doppler waveform analysis results for the common femoral arteries. MR angiographic results for segments examined with time-of-flight or phase-contrast techniques and without gadolinium enhancement were excluded from the analysis.
|
|
|
|
value was calculated as a measure of agreement between extracted categoric variables in the analysis, and the correlation coefficient (r value) was calculated as a measure of agreement between extracted continuous variables in the analysis. The natural logarithm of the diagnostic odds ratio (D) was calculated as follows: D = ln[(TP x TN)/(FP x FN)]. This value represents a summary measure of the diagnostic performance per study (ie, the discriminatory power of the examination). This was the measure of interest in the summary ROC analysis. The correlation between the natural logarithms of the diagnostic odds ratios as derived from the individual studies by the two authors was calculated to summarize the overall agreement in the data extracted by both authors.
Data Synthesis
Funnel plot.To detect publication bias, (ie, the bias resulting from studies with a positive result being published more often than studies with a negative result), we constructed a funnel plot (44). In a funnel plot, the number of units measured per individual study (arterial segments, in this case) is plotted as a function of the measure of interest (the natural logarithm of the diagnostic odds ratio, in this case). In the absence of publication bias, the data points form a symmetric funnel-shaped distribution, whereas a skewed and asymmetric distribution indicates the presence of publication bias. Funnel-plot symmetry was determined informally by means of visual inspection of the graph.
Weighted pooled analysis.Pooled values for effect size are often calculated by means of a weighted pooling of the individual effect sizes, with weights equal to the reciprocal of the variance of each study. To apply this method, the assumption of homogeneity of effect sizes must hold (45,46). We checked the homogeneity of effect sizes with a statistical test, and only the sensitivity of MR angiography was homogeneous. We therefore used a random effects model, which can accommodate heterogeneous, as well as homogeneous, effect sizes. The pooled values based on this model include an estimated component of variance due to interstudy variation (45,46). We calculated the pooled sensitivity, specificity, and natural logarithm of the diagnostic odds ratio and constructed 95% CIs.
Summary ROC analysis.Summary ROC analysis is a meta-analytic method to summarize and combine the true- and false-positive rates for different diagnostic studies (1214). The method involved development of a regression model with the dependent variable being the natural logarithm of the diagnostic odds ratio from each study and the independent variable being a measure of the positivity criterion (S) of the study (ie, classification of an examination as positive): S = ln[(TP x FP)/(TN x FN)]. With this method, one assumes that the differences in examination performance reported in the literature are due partly to variations in the positivity criterion used by different authors; the regression analysis thus allows one to adjust for these differences. Adjustment for important clinical variables and comparison of examinations are also possible. Examples of meta-analyses that used a summary ROC analysis can be found in De Vries et al (4) and Fleischmann et al (47).
The adjustment for clinical variables is accomplished by including them in the regression model. Inclusion of a dummy variable in the regression analysis for the type of diagnostic examination performed (1 for MR angiography, 0 for duplex US) makes it possible to compare the tests. The regression coefficient of this dummy variable is a measure of the difference in discriminatory power between the examinations. A positive regression coefficient indicates increased discriminatory power for MR angiography relative to that of duplex US, whereas a negative regression coefficient indicates reduced discriminatory power.
To prevent undefined values for diagnostic odds ratios, positivity criteria, and their variances that result from zero values for TP, FN, TN, or FP results, 0.5 was added to each TP, FN, TN, and FP value (13).
We assessed the effect of publication year; continent (North America vs other), mean age (65 years or younger vs older than 65 years), prevalence of diseased segments per study, blinded interpretation of the test result (yes or probably vs no or not reported), blinded interpretation of the reference test result (yes vs no or not reported), inclusion of consecutive patients (yes vs no or not reported), type of imager (Ultramark [ATL] vs Acuson vs other for duplex US; Philips Medical Systems vs Siemens Medical Systems vs other for MR angiography), and dose of gadolinium (in millimoles per kilogram body weight) for MR angiography. A quality score was defined on the basis of the following criteria: blinded interpretation of MR or US images, blinded interpretation of the reference modality results, and inclusion of consecutive patients. Studies fulfilling all criteria were given a quality score of 1; all others were given a score of 0. Also, a four-tiered rating of quality for diagnostic imaging studies, described by Kent et al (48), was determined for each study and analyzed by using three dummy variables.
Owing to missing data, the proportion of patients with intermittent claudication could not be analyzed as a covariate and was, therefore, excluded from all analyses. The proportions of femoropopliteal and infrapopliteal segments were missing for three articles, and only the subset of articles with complete data for the relevant variables was used. Age was not reported in six articles, and another three reported median age instead of mean age. The proportion of male patients was not reported in five articles. The maximum number of days between MR angiography or duplex US and the conventional angiography was not reported in four articles. Estimates of missing values were calculated (imputed) with weighted means or with a best-subset regression analysis (Stata reference manual, vol 2, release 4.0; College Station, Tex: Stata Statistical Software) and were used to explore the effect of clinical variables by using all the available articles to ensure that no potentially important covariate was overlooked.
MR angiography and duplex US were first analyzed separately and were subsequently compared in one model. Univariate analyses were performed to assess the effect of each clinical covariate by using the subset of articles with complete data for that variable and estimated values if necessary. The latter were used to select variables for the final analysis only if no or minor differences in the results between the subset of articles with complete data and the calculation with estimated values were found. Significant variables (P < .05) were subsequently assessed in a multivariate model per examination. Variables with a P value greater than .05 but less than .10 were retained in the model if the explanatory power of the model increased substantially (adjusted R2 increased by 0.05).
In the final multivariate analysis in which MR angiography and duplex US were compared, missing values for significant variables in the model were not substituted with estimates, and only the subset of articles with complete data for the selected variables was used. The selection of variables for the final model was performed by analyzing each variable in turn and considering significant variables alone, so as to avoid exclusion of as few studies as possible. Also, interaction terms were incorporated in this analysis to allow for differential effects, depending on the examination. We tested for heterogeneity across studies by comparing the 95% CI of the observed values of the natural logarithm of the diagnostic odds ratio in each study with that of the predicted values of the diagnostic odds ratio by using the final model. Finally, we re-analyzed the final summary ROC model with a random effects regression analysis (Stata technical bulletin no. 42, College Station, Tex: Stata Statistical Software), which took interstudy variability into account. All analyses were performed with SPSS for Windows (release 7.5.2; SPSS, Chicago, Ill) and STATA (release 5.0, Stata Statistical Software) software.
Sensitivity analyses.A so-called jackknife type of sensitivity analysis of the final model was accomplished by performing multiple summary ROC analyses and excluding each article in turn. The jackknife sensitivity analysis is used to determine the contribution of the results in each article to the overall analysis. Three articles on duplex US used another definition of disease as determined at conventional angiography: Instead of 50% stenosis, 70% stenosis was used in one study (27), and severe vessel wall irregularities were used in two others (31,32). In a sensitivity analysis, with dummy variables for the studies with another definition of disease, we determined whether these studies demonstrated a difference in diagnostic performance as compared with the other studies on duplex US. As before, a positive coefficient indicated increased discriminatory power, and a negative coefficient indicated reduced discriminatory power. In one study on MR angiography (26), the authors reported that gadolinium-based contrast material was administered only in selected patients, and we evaluated the results in this study by using a dummy variable for comparison with the other MR angiography studies.
| RESULTS |
|---|
|
|
|---|
Overall, 38 of 372 (10%) discrepancies in data extraction occurred between the two authors and ranged from zero to 12 of 31 (0%39%) for the different variables (Table 5). All discrepancies were resolved by means of consensus. In Tables 1 and 2, the examination characteristics of the 31 studies included in the meta-analysis are outlined; Tables 3 and 4 present the clinical characteristics. All studies but one (25) were judged to have been prospective. The MR angiographic results in 216 patients (from nine articles) were included in the baseline analysis. The mean age among patients who underwent MR angiography was 63 years; 72% were men, and 28% were women. Duplex US results reported in 18 studies with 1,059 patients were included in the baseline analysis. The mean age among patients who underwent duplex US was 65 years; 65% were men, and 35% were women. The funnel plot (Fig 1) for the number of analyzed segments as a function of the discriminatory power of the study (natural logarithm of the diagnostic odds ratio) demonstrated a symmetric funnel-shaped distribution for the duplex US studies, which suggests that publication bias was unlikely to be present. Although the MR angiographic studies were symmetrically distributed, there were a few with a large number of segments; therefore, publication bias could not be properly evaluated.
|
|
Summary ROC Analysis
No significant predictors were demonstrated in the univariate analysis for MR angiography, and no effect was demonstrated for different positivity criteria (regression coefficient, 0.16 [95% CI: -0.66, 0.98]; P = .65) The univariate analysis for duplex US demonstrated that as the maximum number of days between duplex US and conventional angiography increased, the discriminatory power of duplex US decreased [regression coefficient, -0.031 per day [95% CI: -0.060, -0.002]; P = .04). In the same model, the regression coefficient for the positivity criterion of duplex US was a significant predictor [regression coefficient, -0.66 [95% CI: -1.14, -0.19]; P = .01). For both MR angiography and duplex US, neither the quality scores nor the individual covariates for evaluating individual aspects of quality were significant predictors of diagnostic performance.
In the comparison analysis, without adjustment for covariates, the discriminatory power of MR angiography (regression coefficient, 2.11 [95% CI: 0.12, 4.09]) was better than that of duplex US (P = .04). When we adjusted for each covariate in turn, the discriminatory power of MR angiography again was better than that of duplex US, with regression coefficients of 1.182.27. In the comparison analysis, the time between duplex US and conventional angiography and the positivity criterion were significant predictors, with similar coefficients as in the analysis for duplex US examination alone. Age had a significant effect on discriminatory power, with a regression coefficient of -0.13 per year (95% CI: -0.250, -0.003; P = .05).
In the final model with adjustment for multiple covariates, the discriminatory power of MR angiography was better than that of duplex US but with a slightly lower regression coefficient of 1.67 (95% CI: -0.23, 3.56) that only approached statistical significance (P = .08). The interaction terms were included to allow for a duplex USspecific effect for the maximum number of days between duplex US and conventional angiography (regression coefficient, -0.031 per day [95% CI: -0.055, -0.006]; P = .02) and for the positivity criterion (regression coefficient, -0.63 [95% CI -1.05, -0.21]; P = .005) (Table 6). Age was not a significant predictor in the multivariate model. The adjusted R2 value for the final model was 0.41. Figure 2 presents the summary ROC curves determined on the basis of the final model for MR angiography and duplex US, adjusted to 0 days between duplex US and conventional angiography.
|
|
Sensitivity Analyses
Exclusion of all articles, one by one, from the final model did not have a large effect on the difference in discriminatory power between MR angiography and duplex US (regression coefficient range, 1.332.14). In a sensitivity analysis, we included three duplex US studies in which another definition of disease or another positivity criterion was used. The study with another percentage of diameter reduction as the definition of disease (27) demonstrated better discriminatory power, but this difference was not significant in comparison with results from the other duplex US studies (regression coefficient, 2.66 [95% CI: -6.93, 12.25]; P = .56). The two studies by Koelemay et al (31,32) used irregularities of the vessel wall and luminal narrowing seen at duplex US as positivity criteria and severe vessel wall irregularities seen at conventional angiography as the definition of disease. Results from these two studies demonstrated a discriminatory power that was lower than those of the other duplex US studies (regression coefficient, -2.16 [95% CI: -3.21, -1.10]; P = .001). In the study by Cambria et al (26), a gadolinium-based contrast agent was administered only in selected patients, which decreased the discriminatory power relative to that of the other MR angiography studies (regression coefficient, -2.20 [95% CI: -3.92, -0.49]; P = .02).
| DISCUSSION |
|---|
|
|
|---|
A limitation of pooling sensitivities and specificities is that different positivity criteria used in individual studies are not taken into account. Summary ROC analysis, on the other hand, allows adjustment for different positivity criteria. Furthermore, summary ROC analysis can be extended to a multivariate regression analysis to adjust for differences in patient characteristics, study design, and diagnostic imaging protocols. Adjustment for these differences was limited in our analysis, however, because not all articles included the relevant details. For example, the mean age of patients was not always reported. The results based on comparative summary ROC analysis of the subset of studies in which mean age was reported suggested that both types of studies had lower discriminatory power in elderly patients, but firm conclusions about the effect of age cannot be made. Furthermore, the percentage of patients with intermittent claudication and the percentages of male and female patients could not be extracted from all articles. Finally, although most of the duplex US articles reported site-specific results, only five of the nine MR angiography articles did so, which made a comparative subgroup analysis by anatomic site impractical. We did, however, adjust for the proportion of femoropopliteal and infrapopliteal segments in our analysis and found no difference in effect with this adjustment.
As with all meta-analyses, the present study was limited by the ambiguity of the originally reported data. This ambiguity can lead to differences in interpretation and discrepancies in extraction of data. To minimize the bias due to this limitation, both authors extracted the data independently. Discrepancies occurred in 10% of the extracted data points. Overall, the authors demonstrated good agreement with regard to the extracted information, with
values ranging from 0.63 to 1.00 and correlation coefficients (r values) ranging from 0.92 to 1.00.
Another limitation of meta-analyses is that the quality of the original studies may affect the results. L'Abbé et al (49) recommended that a quality score be calculated for studies included in a meta-analysis. In our analysis, we evaluated the quality of the study design and reporting methods by distinguishing studies that fulfilled all quality criteria versus those that did not and by using a published rating score (48) for the quality of diagnostic imaging studies. In the summary ROC analysis, neither the quality scores nor the individual covariates that constituted the quality score were significant predictors of diagnostic performance.
Publication bias may have affected our results. Although we demonstrated that publication bias was unlikely to be present for duplex US studies, we cannot exclude such bias, because the limited number of data points could have decreased the power of detecting publication bias with the funnel plot. Moreover, the limited number of MR angiography studies, all of which included relatively small numbers of patients, did not enable us to detect whether publication bias was present. Another potential issue related to publication bias is that early reports of diagnostic test results generally are more favorable. This could have potentially biased the results against duplex US, because publications on duplex US have appeared since the early 1980s, whereas gadolinium-enhanced MR angiography was first performed in the early 1990s. To determine whether this may have influenced the results, we evaluated the effect of publication year and found that the discriminatory power of MR angiography and duplex US did not change over the years.
In general, the time between MR angiography and conventional angiography was shorter (
14 days; mean, 5 days) than that between duplex US and conventional angiography (
61 days; mean, 17 days). With an increase in the period between the comparison examination (MR angiography or duplex US) and the reference standard examination (conventional angiography), one would expect a greater change in the disease status due, for example, to progression of disease (50). Thus, with an increase in the time between the comparison and the reference examinations, one can expect a decrease in the discriminatory power of the comparison examination, which was indeed demonstrated in this analysis for duplex US. In fact, the lower discriminatory power of duplex US as compared with MR angiography was explained in part by the longer time between duplex US and conventional angiography relative to that between MR angiography and conventional angiography. This implies that the comparison and reference examinations should ideally be performed on the same day and that comparisons between MR angiography and duplex US should take this factor into account, which we did by adjusting for this variable in the final model.
The reported sensitivities for duplex US span a broad range, which may be explained by the fairly long delay between US and conventional angiography (discussed in the preceding paragraph), operator dependency, differences in technique, variation among duplex US machines, or differences in patient populations. Results from a previous study (51) in which US assessment of carotid arterial stenosis was evaluated suggested that differences in the hardware or software of duplex US machines could cause discrepancies in measured velocities. We could not, however, detect a difference in diagnostic performance for different duplex US machines.
Questions have been raised concerning the use of conventional angiography as the reference standard for comparison with MR angiography. Owen et al (52) and Carpenter et al (53) reported that conventional angiography may not be a good reference method for demonstrating runoff vessels, because MR angiography demonstrated more patent vessels than did conventional angiography. Thus, some authors (26,42) reported a measure of agreement (
statistic) rather than the sensitivity and specificity for the comparison of MR angiographic results with conventional angiographic results. Cambria et al (26) reported
values of 0.480.60, and Quinn et al (42) reported substantial to perfect agreement (
= 0.611.00) for nearly all anatomic segments.
The authors of three articles on MR angiography (8,10,38) compared different techniques for performing the examination. In our meta-analysis, we included only the results of gadolinium-enhanced MR angiography because gadolinium enhancement has rapidly become the standard and has been shown to substantially improve the diagnostic performance of MR angiography (8,38,54). With the ongoing technical developments in MR imaging, one can expect future improvements in the discriminatory power of MR angiography. Major improvements in duplex US are less likely, although the use of intravascular contrast agents could potentially improve this method, as well. Such improvements may necessitate an update of the meta-analysis in the future.
Ideally, MR angiography and duplex US should be compared in the same group of patients or should be randomly assigned to groups of patients (14). The literature search did not retrieve any articles in which gadolinium-enhanced MR angiography was compared directly with color-guided duplex US. It is possible that the demonstrated differences in our analysis of diagnostic performance between MR angiography and duplex US may reflect differences in study or patient characteristics. In fact, the current results suggest that part of the superiority in diagnostic performance of MR angiography, as compared with that of duplex US, was explained by such differences. Furthermore, it should be noted that both examinations were performed in highly selected patient populations. Almost all studies were conducted in an academic setting in either North America or Europe and included patients usually scheduled for preinterventional work-up for peripheral arterial disease. Widespread use of MR angiography and duplex US in patients with broader clinical indications might result in different diagnostic performance of these examinations; therefore, generalization of our results should be made with caution.
The goal of imaging in the work-up for peripheral arterial disease is not to discriminate between patients with and those without disease (the history and ankle-brachial index do that), but rather to discriminate between diseased and nondiseased segments, that is, to localize the disease. Thus, in determining the sensitivity and specificity of imaging examinations for peripheral arterial disease, data analysis according to segment probably is the most relevant, and, in all articles but one (16), results were indeed reported according to segment rather than according to limb or patient. This would imply that multiple segments per patient were analyzed in each study and that observations within each study may have been correlated. Correlation of the observations within each study is a characteristic of the individual study results and does not imply that the data points used in a meta-analysis are correlated. In a meta-analysis, potential bias related to this problem can be adjusted if the correlations in the individual studies are known, which generally is not the case. Nevertheless, the fact that disease severity in arterial segments is correlated within a patient does not necessarily mean that the diagnostic performance is correlated as well.
Ultimately, diagnostic information according to segment must be integrated to enable treatment decision making for the patient. Evaluation and optimization of the decision-making process will require an extensive analysis that must take into account not only the diagnostic accuracies of the examination, as presented here, but also the effects of imaging on treatment planning, prognosis, quality of life, local expertise, availability of equipment, and costs.
In conclusion, our results suggest that the discriminatory power of gadolinium-enhanced MR angiography was better than that of color-guided duplex US and that MR angiography was a highly sensitive and specific method, as compared with conventional angiography, for the work-up for peripheral arterial disease, which implies that MR angiography could potentially replace duplex US and conventional angiography.
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
Author contributions: Guarantor of integrity of entire study, M.G.M.H.; study concepts and design, M.G.M.H., K.V.; definition of intellectual content, M.G.M.H., K.V.; literature research, K.V.; data acquisition, M.G.M.H., K.V.; data analysis, K.V.; statistical analysis, M.G.M.H., K.V.; manuscript preparation, K.V.; manuscript editing, M.G.M.H., K.V.; manuscript review, M.G.M.H.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
L.R. Arends, T.H. Hamza, J.C. van Houwelingen, M.H. Heijenbrok-Kal, M.G.M. Hunink, and T. Stijnen Bivariate Random Effects Meta-Analysis of ROC Curves Med Decis Making, September 1, 2008; 28(5): 621 - 638. [Abstract] [PDF] |
||||
![]() |
T. H. Hamza, J. B. Reitsma, and T. Stijnen Meta-Analysis of Diagnostic Studies: A Comparison of Random Intercept, Normal-Normal, and Binomial-Normal Bivariate Summary ROC Approaches Med Decis Making, September 1, 2008; 28(5): 639 - 649. [Abstract] [PDF] |
||||
![]() |
R. Ouwendijk, M. de Vries, T. Stijnen, P. M. T. Pattynama, M. R. H. M. van Sambeek, J. Buth, A. V. Tielbeek, D. A. van der Vliet, L. J. SchutzeKool, P. J. E. H. M. Kitslaar, et al. Multicenter Randomized Controlled Trial of the Costs and Effects of Noninvasive Diagnostic Imaging in Patients with Peripheral Arterial Disease: The DIPAD Trial Am. J. Roentgenol., May 1, 2008; 190(5): 1349 - 1357. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Thurnher, S. Miller, G. Schneider, C. Ballarati, G. Bongartz, C. U. Herborn, S. Schoenberg, M. A. Cova, G. Morana, K. Niazi, et al. Diagnostic Performance of Gadobenate Dimeglumine Enhanced MR Angiography of the Iliofemoral and Calf Arteries: A Large-Scale Multicenter Trial Am. J. Roentgenol., November 1, 2007; 189(5): 1223 - 1237. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. H. Heijenbrok-Kal, M. C. J. M. Kock, and M. G. M. Hunink Lower Extremity Arterial Disease: Multidetector CT Angiography Meta-Analysis Radiology, November 1, 2007; 245(2): 433 - 439. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Albrecht, E. Foert, R. Holtkamp, M. A. Kirchin, C. Ribbe, F. K. Wacker, M. Kruschewski, and B. C. Meyer 16-MDCT Angiography of Aortoiliac and Lower Extremity Arteries: Comparison with Digital Subtraction Angiography Am. J. Roentgenol., September 1, 2007; 189(3): 702 - 711. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. de Vries, R. Ouwendijk, K. Flobbe, P. J. Nelemans, A. G. Kessels, G. H. Schurink, J. A. van der Vliet, F. M. J. Heijstraten, P. W. M. Cuypers, L. E. M. Duijm, et al. Peripheral Arterial Disease: Clinical and Cost Comparisons between Duplex US and Contrast-enhanced MR Angiography--A Multicenter Randomized Trial. Radiology, August 1, 2006; 240(2): 401 - 410. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Thompson and R. House Hand-arm vibration syndrome with concomitant arterial thrombosis in the hands Occup. Med., August 1, 2006; 56(5): 317 - 321. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. W.S. Rutjes, J. B. Reitsma, M. Di Nisio, N. Smidt, J. C. van Rijn, and P. M.M. Bossuyt Evidence of bias and variation in diagnostic accuracy studies. Can. Med. Assoc. J., February 14, 2006; 174(4): 469 - 476. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. FURLOW Aortic Emergencies Radiol. Technol., September 1, 2005; 77(1): 41 - 49. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Ota, K. Takase, H. Rikimaru, M. Tsuboi, T. Yamada, A. Sato, S. Higano, T. Ishibashi, and S. Takahashi Quantitative Vascular Measurements in Arterial Occlusive Disease RadioGraphics, September 1, 2005; 25(5): 1141 - 1158. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. S. Gazelle, P. M. McMahon, U. Siebert, and M. T. Beinfeld Cost-effectiveness Analysis in the Assessment of Diagnostic Imaging Technologies Radiology, May 1, 2005; 235(2): 361 - 370. [Abstract] [Full Text] [PDF] |
||||