| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Vascular and Interventional Radiology |
1 From the Departments of Epidemiology (P.J.N., H.C.W.d.V.) and Radiology (T.L., J.M.A.v.E.), University of Maastricht, P Debyeplein 1, 6229 HA Maastricht, the Netherlands; and the Cochrane Methods Group on Systematic Review of Screening and Diagnostic Tests (H.C.W.d.V.). Received November 24, 1998; revision requested January 26, 1999; final revision received January 18; accepted February 7. Address correspondence to P.J.N. (e-mail: patty.nelemans@epid.unimaas.nl).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
MATERIALS AND METHODS: A search strategy in MEDLINE and citation tracking were used to identify relevant English-language articles published since 1991. Each article was critically appraised for examination, patient, and study design characteristics. The accuracy data from different studies were analyzed by constructing summary receiver operating characteristic curves; multiple linear regression was used to examine the variation between study results.
RESULTS: Twenty-three studies were included. There was much heterogeneity in the study results, which could not be explained as differences in the threshold for a positive result. About half of the variation was due to the type of MR angiographic examination and the extent of image evaluation. The relative diagnostic odds ratio (DOR) for three-dimensional (3D) gadolinium-enhanced MR angiography compared with two-dimensional (2D) time-of-flight MR angiography was 7.46 (95% CI: 2.48, 22.20). The relative DOR for review of transverse source images or multiplanar reformations in addition to maximum intensity projections (MIPs) compared with the use of only MIPs for image evaluation was 4.53 (95% CI: 1.46, 13.87).
CONCLUSION: The diagnostic accuracy of 3D gadolinium-enhanced MR angiography is superior to that of 2D time-of-flight MR angiography. Also, the review of transverse source images or use of additional postprocessing techniques, such as multiplanar reformation, results in significantly better diagnostic performance.
Index terms: Arteries, MR, 91.12942, 91.12943, 92.12942, 92.12943 • Arteries, peripheral, 91.721, 92.721 • Arteries, stenoses or obstruction, 91.721, 92.721 • Arteriosclerosis, 91.721, 92.721 • Magnetic resonance (MR), comparative studies • Magnetic resonance (MR), technology • Receiver operating characteristic (ROC) curve
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
Over the years, multiple reports have been published that validate MR angiography against conventional angiography as the standard. As is often the case in the evaluation of diagnostic examinations, the various studies yielded a broad spectrum of values for sensitivity and specificity. Meta-analysis has become popular as a method for summarizing results from independent studies and for exploring the potential sources of variation between study results. The meta-analysis of diagnostic accuracy is a relatively new area of investigation, but several methods have been proposed.
It was common practice in medical review articles to present means of the sensitivity and specificity estimates weighted by the sample size of the studies. However, such pooling relies on a single threshold or cutoff point for classifying an examination result as positive. Often, this assumption is not justified. Even when studies use the same explicit threshold, their implicit threshold may differ, especially if interpretation of the images requires judgment. The use of implicit thresholds is common in the interpretation of radiologic imaging techniques. Radiologists may agree to use the same words to describe imaging examination results but still differ in what they regard as the boundary between "abnormal" and "probably normal" (1). This problem may be less pertinent to MR angiography, because the degree of stenosis is measured semiquantitatively. However, in some situations, for example, in stenoses between 45% and 55%, the ultimate decision of whether the stenosis is smaller or larger than 50% may still require some judgment.
A change in the threshold for a positive examination result that increases the sensitivity leads to a decrease in the specificity and vice versa. This trade-off between sensitivity and specificity makes it imperative that they be considered jointly. In meta-analysis, this is possible with construction of a summary receiver operating characteristic (ROC) curve (2,3).
The present meta-analysis has the purposes of (a) summarizing the overall diagnostic performance of MR angiography in the detection or exclusion of stenoses of 50%–99% or occlusions in the peripheral vascular tree and (b) identifying the most important sources of variation in diagnostic accuracy between studies. Study results are summarized with summary ROC curves.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
The search terms used were peripheral vascular disease; peripheral arterial disease; arterial occlusive disease; intermittent claudication; arterial insufficiency; lower limb ischemia; lower extremity ischemia; peripheral; lower extremity combined with magnetic resonance angiography, MR angiography, or MRA. Additional articles were obtained with citation tracking of review articles and original articles.
Only original studies were included. The other inclusion criteria were (a) conventional angiography was used as the standard of reference; (b) a hemodynamically significant lesion was defined as either a stenosis of 50%–99% or an occlusion; (c) the absolute numbers of true-positive (TP), false-negative (FN), true-negative (TN), and false-positive (FP) observations were available or derivable from the data presented.
Data Extraction
A standard form was used to extract relevant data from the included studies. Recorded were the examination, patient, and study design characteristics and the examination results.
Examination characteristics.—A distinction was made between studies in which the diagnostic accuracy of two-dimensional (2D) time-of-flight MR angiography was evaluated and those in which the diagnostic accuracy of three-dimensional (3D) gadolinium-enhanced MR angiography was evaluated. The year of publication was considered to be an indicator of advances in MR angiographic technology and learning experience associated with the generation and interpretation of MR angiographic images. Also documented were the extent of image evaluation, that is, only maximum intensity projection (MIP) versus MIP supplemented by review of transverse source images or additional postprocessing techniques, and whether cardiac synchronization was used for 2D time-of-flight MR angiography.
Patient characteristics.—For each study population, the mean age; percentage of male patients; percentage of patients with clinical indications such as claudication, critical ischemia, or other indications; and anatomic sites studied were recorded. Also documented was the percentage of aortoiliac segments included in the evaluated trajectory. With 2D time-of-flight MR angiography, the aortoiliac segments are more difficult to image, because the tortuous course of these vessels can cause partial in-plane saturation and diminish signal intensity. This loss of signal intensity sometimes mimics stenosis, which results in FP results.
Study design characteristics.—Characteristics of the study design considered important were blinding of the readers of MR angiographic images to the results of conventional angiography and vice versa (yes vs no), sample size, and a low potential for verification bias (yes vs no). Verification bias may exist if the decision to perform the standard examination depends on the results of the examination under investigation, that is, if verification with conventional angiography occurred more often in patients with positive MR angiographic results than in patients with negative MR angiographic results (4).
Examination results.—In many articles, the numbers of TP, FN, TN, and FP observations were directly available. If not, the numbers were derived from the marginal totals and the reported sensitivity and specificity. Where possible, site-specific results were also listed. For studies in which more than one anatomic level was evaluated, the data were pooled into three anatomic areas: the aortoiliac arteries, the femoropopliteal arteries, and the below-knee arteries. If results were reported for more than one observer, the results from the first observer were used.
Data Analysis
Summary ROC curves.—For fitting a summary ROC curve, Littenberg and Moses (2) and Moses et al (3) proposed a data-analytic approach in which linear regression on the logit scale was used. This method includes three steps. In ROC analysis, sensitivity, or the TP rate (TPR), is plotted on the vertical axis against the corresponding FP rate (FPR, 1 - specificity) on the horizontal axis. The first step is to transform the vertical and horizontal scales in a way that allows for fitting of a straight-line regression, because linear regression models can be fit within statistical packages that are widely available. For this purpose, the sensitivity and the FPR of each study are transformed to the logit scale. The logit of a probability p is the natural logarithm of the odds of p, that is, logit p = ln[p/(1 - p)]. Following this logit transformation, two new variables D and S are constructed, where D is the difference between and S is the sum of the logit transforms: D = logit(sensitivity) - logit(1 - specificity), and S = logit(sensitivity) + logit(1 - specificity).
The dependent variable D is equal to the natural logarithm of the diagnostic odds ratio (DOR): ln (DOR). The DOR is a measure of the discriminatory power of an examination and represents the odds of a positive examination result among diseased persons relative to the odds of a positive examination result among nondiseased persons (see the Appendix for a more detailed explanation). S is a measure of the threshold for classifying an examination result as positive; as the threshold becomes more lenient (less stringent), both sensitivity and FPR become larger, so S increases.
The second step involves plotting the D values from each study on the vertical axis against the corresponding S values on the horizontal axis. In this plot, a linear regression line D = A + BS is fitted. The y intercept A of the model is the estimated ln DOR when S is 0, and the regression coefficient B provides an estimate of the extent to which the ln DOR is dependent on the threshold (5).
In the third step, the transformation is reversed to find the summary ROC curve. Once the slope and the intercept A of the transformed line are known, a summary ROC curve can be constructed by means of back transformation with the equation TPR = 1/
{1 + exp[-A/(1 - B)]} x [(1 - FPR)/FPR](1+B)/(1-B)
.
Remaining sources of between-study variation.—The described method is useful for evaluating whether the variation between study results can be explained as differences in the threshold for a positive examination result. Furthermore, this data-analytic approach by using linear regression permits the identification of other important sources of variation in diagnostic accuracy between studies.
The model D = A + BS can easily be expanded to a multiple linear regression model by adding one or more covariates, such as examination (X1), patient (X2), and study design (X3) characteristics: D = A + BS + B1X1 + B2X2 + B3X3. The regression coefficients (B1, B2, B3) are indicators of the independent effects of the corresponding covariates (X1, X2, X3) on the dependent variable, that is, ln DOR. The magnitude of the regression coefficient of a variable represents the difference in ln DOR between studies with different levels of that variable, with all other variables held constant. A large regression coefficient indicates that the corresponding covariate has a large influence on diagnostic accuracy. After antilogarithmic transformation, the regression coefficient can be interpreted as a relative DOR.
The basic regression model D = A + BS was first expanded by adding one covariate. Separate analyses were performed for nine potentially relevant study design characteristics, which were entered as dichotomous variables with values of 1 or 0: MR angiographic examination type (3D gadolinium enhanced versus 2D time of flight), use of cardiac synchronization (yes vs no), year of publication (after 1995 vs 1995 or before), postprocessing technique (MIP in combination with review of transverse source images or multiplanar reformation vs only MIP), mean age (
65 years vs <65 years), percentage of male patients (75% vs <75%), the prevalence of stenosed segments (>25% vs 
25%), the presence of aortoiliac segments in the evaluated trajectory (yes vs no), and sample size (>30 vs 30). Studies with missing values for one or more of the variables were excluded from the relevant analyses.
The relative importance of these nine study design characteristics was assessed according to the magnitude of the corresponding relative DOR. The examination, patient, and study design characteristics with the largest relative DORs were evaluated simultaneously in one multiple variable model to evaluate the independent effects on the ln DOR.
The goodness of fit of the regression line D = A + BS is evaluated by using R2, which is the square of the correlation coefficient for the relationship between the observed values of D and the predicted values of D, and provides a quantitative measure of how predictive of the dependent variable the combination of independent variables in the model is. If all the observed values fall on the fitted regression line, R2 is 1. If there is no linear relationship between the observed and predicted values, R2 is 0 (5). The linear regression models were fitted by using unweighted least squares analysis, that is, by giving equal weight to each study. Weighted analysis—weighting by the inverse of the variance of each study—is inappropriate, because it gives more weight to studies that appear to show poorer accuracy (6).
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
A list of studies that were excluded from meta-analysis and the reasons for exclusion is available on request. The most important reasons for exclusion were that conventional angiography was not used as the standard, another cutoff point was used, a measure of agreement between MR angiographic and conventional angiographic findings other than sensitivity and specificity was used, and/or the absolute numbers of TP, FN, TN, and FP observations could not be derived. Also excluded were articles in which 2D gadolinium-enhanced or phase-contrast MR angiography was evaluated, because the number of articles on these specific MR angiographic examinations was too small to enable a meta-analytic comparison with articles on 2D time-of-flight MR angiography or 3D gadolinium-enhanced MR angiography.
Table 1 gives an overview of the examination and patient characteristics in the included studies. In total, 344 patients were evaluated in 13 studies on 2D time-of-flight MR angiography and 253 patients were evaluated in 10 studies on 3D gadolinium-enhanced MR angiography.
|
|
|
|
|
Table 4 shows the regression coefficients and relative DORs for nine potentially relevant study design characteristics. The three characteristics with the largest relative DORs were MR angiographic examination type, year of publication, and extent of image evaluation. These covariates and the variable S were entered simultaneously into one multivariate model. We kept the variable S in the model, because capturing its effect on D is essential to the data-analytic approach used in this meta-analysis. The definition of a large relative DOR (>2) is arbitrary, but it was decided to add no more than three covariates to the basic model D = A + BS because of the limited number of studies included in the meta-analysis (n = 23). The extent of image evaluation (postprocessing technique) was associated with a relative DOR exceeding 2, and although this relative DOR was not statistically significant (95% CI: 0.70, 12.43), the a priori hypothesis that review of transverse source images or use of additional postprocessing techniques increases diagnostic performance was considered strong enough to include this covariate in the multivariate model.
|
About half of the between-study variation was due to four factors: (a) the intercept A; (b) the variable S, which is a measure of the leniency of the threshold for a positive examination result; (c) the MR angiographic examination type; and (d) the extent of image evaluation. The adjusted relative DORs and corresponding 95% CIs for the MR angiographic examination type and the extent of image evaluation that result from the final model are presented in Table 4. The relative DOR for 3D gadolinium-enhanced MR angiography (compared with 2D time-of-flight MR angiography) was 7.46 (95% CI: 2.48, 22.20); the relative DOR for use of only MIP compared with MIP plus review of transverse source images or additional postprocessing techniques was 4.53 (95% CI: 1.46, 13.87).
Other factors, such as the inclusion of aortoiliac segments in the evaluated trajectory, the prevalence of stenosed segments, a high percentage of male patients, a high mean age of the study population, the use of cardiac synchronization, and a large sample size were associated with only small relative DORs (Table 4). The results indicate that these study design characteristics are poor predictors of diagnostic performance. Clinical indications could not be included in the regression model, because in seven of the 23 studies this information was lacking.
Figure 2a plots the regression lines (predicted D vs S) for four groups of studies chosen to be the available combinations of the two covariates MR angiographic examination type and extent of image evaluation. Group 1 comprises studies on 2D time-of-flight MR angiography with only MIP as a postprocessing technique (12–16,26), group 2 comprises studies on 2D time-of-flight MR angiography with MIP plus review of transverse source images or multiplanar reformations as a postprocessing technique (7,9–11,17,27), group 3 comprises studies on 3D gadolinium-enhanced MR angiography with only MIP as a postprocessing technique (19–22,24–26), and group 4 comprises studies on 3D gadolinium-enhanced MR angiography with MIP plus review of transverse source images or multiplanar reformations as a postprocessing technique (18,23,27).
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
It was not possible to give an adequate summary of all study results with one common underlying summary ROC curve for each MR angiographic examination. On the basis of the results of linear regression analysis, separate summary ROC curves were constructed for four subgroups. Within these subgroups, the summary ROC curves provided a more adequate description of the study results.
It should be kept in mind that a summary ROC curve differs from traditional ROC analysis. A traditional ROC curve represents a single population and describes how the sensitivity and the TPR vary as the threshold for a positive examination result varies, all else being held constant. A summary ROC curve results from fitting a smooth line near data points that represent pairs of sensitivity and FPR from multiple studies, which shows differences with respect to factors affecting sensitivity and specificity estimates.
The presentation of the summary ROC curve should be restricted to the range of sensitivities and FPRs reported in the included studies. Therefore, the area under the curve, which is a global measure of diagnostic accuracy in traditional ROC analysis, is not available for the summary ROC curve (3).
An important advantage of a summary ROC curve as a summary of diagnostic accuracy across studies is that it provides a description of all pairs of sensitivity and FPR. In clinical decision-making and cost-effectiveness analyses, the summary ROC curve can provide data for modeling the effect of examination on patient treatment and clinical outcomes at different operating points.
It has been hypothesized that the accuracy of MR angiography may vary with the anatomic level, with a higher accuracy of 2D time-of-flight MR angiography at lower anatomic levels (below-knee > femoropopliteal > aortoiliac) (30). With 3D gadolinium-enhanced MR angiography, the situation may be the other way around. Distal vessels are more difficult to study, because the concentration of contrast material is reduced or the arrival of the bolus may be delayed, particularly if proximal vessels are diseased (22). Also, veins filled with contrast material can cause diminished image interpretability because of overprojection.
Unfortunately, a meta-analysis of site-specific results was infeasible, because for 10 of 23 studies it was not possible to classify the studied anatomic sites into mutually exclusive anatomic levels, such as aortoiliac versus femoropopliteal versus below-knee arteries (Table 3). Site-specific results for more than one anatomic level could be derived from three 2D time-of-flight MR angiography studies (11,16,17) and from two 3D gadolinium-enhanced MR angiography studies (20,24). These studies showed no clear trend in diagnostic accuracy with varying anatomic level (Table 3).
In many respects, the method in the included studies was adequate. Most studies provided a detailed description of the MR angiographic technique and the method of conventional angiography. In almost all studies, the readers of MR angiographic and conventional angiographic images were blinded to the results of the other imaging technique.
However, information on clinical indication was lacking in seven of 23 studies. Such information is important, because it describes the distribution of severity of disease in the patients selected for study and the sensitivity and specificity may be higher in more severely affected patients. Therefore, part of the unexplained variation between studies may be due to differences in patient selection.
Many studies also did not provide the information needed to exclude the potential for verification bias. This bias usually results in overestimation of sensitivity and underestimation of specificity. The best way to avoid verification bias is to perform a prospective study with consecutive patients in which all patients receive definite verification of disease status or to examine a random sample of consecutive patients (4).
Nonconsecutive patients were included in three studies (15,16,19), and as many as 10 studies lacked information on this item (Table 1). Only two studies gave a detailed description of the number of eligible patients, the number of excluded patients, and the reasons for exclusion (11,23). Therefore, it was not possible to study the effect of verification bias on diagnostic accuracy. The effects of verification bias are worse as the diagnostic examination under study becomes more accurate, because physicians begin to rely on the examination more as a way to screen for those who need the verification examination.
In this meta-analysis, we used conventional angiography as the standard of reference and focused on the ability of MR angiography to depict hemodynamically significant stenoses. However, for selection of the optimal surgical treatment and distal runoff vessels, the ability to detect patent vessels is also important. For this purpose, the use of conventional angiography as the standard of reference has been discussed. Several reports documented better visualization of patent vessel segments at MR angiography; therefore, conventional angiography was not considered suitable as a standard. These reports came from a single group, and the ability of 2D time-of-flight MR angiography to show patent runoff vessels not revealed at conventional angiography varied between 13% and 23% (14,29,31,32).
Two studies (28,30) compared both MR angiography and conventional angiography with intraoperative angiography as the standard of reference. Both studies were excluded from this meta-analysis for two reasons. First, it was preferred to restrict the meta-analysis to studies with the same standard of reference. Inclusion of studies with another examination standard would add another potential source of heterogeneity to the study results. Second, intraoperative angiography can be used only in selected study populations, such as patients who are candidates for bypass surgery. Moreover, the excluded studies by Baum et al (28) and Huber et al (30) focused on the use of MR angiography to detect patent vessels rather than stenosed vessels.
A possible limitation of this meta-analysis is that multiple regression analysis was used with many (nine) covariates to analyze 23 studies. Several multiple regression models that included different covariates were explored, and eventually the most parsimonious model with a good fit to the observed data was selected. Multiple examinations could have resulted in overestimation of the statistical significance. On the other hand, due to the relatively small number of studies, the relative DORs for most study design characteristics had wide 95% CIs.
Another important problem in the performance of meta-analyses is the possibility of publication bias. Studies that eventually get published are likely to be a biased set, probably with overestimation of examination accuracy (6). In particular, small studies that show overly optimistic results may be published more easily than small studies with less favorable results. However, evaluation of the effect of sample size on the DOR showed that small studies (<30 patients) did not show a better diagnostic performance D than larger studies.
It is concluded that one of the objectives of this meta-analysis, to summarize the results of all studies with one summary ROC curve, could not be achieved because of heterogeneity in the studies. Half of the variation between study results was explained by differences in the evaluated MR angiographic examination and differences in the extent of image evaluation. The other part of the variation remained unexplained, probably because of method problems that could not be captured by the evaluated examination, patient, and study design characteristics. The higher overall diagnostic accuracy of studies in which 3D gadolinium-enhanced MR angiography was evaluated compared with studies in which 2D time-of-flight MR angiography was evaluated supports the use of 3D gadolinium-enhanced MR angiography in the evaluation of peripheral vascular disease. Also, review of transverse source images or supplementation of MIP with additional postprocessing techniques results in better diagnostic performance.
|
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
In Table A1, n1 is the number of persons with disease, and n2 is the number of persons without disease; a is the number of TP observations, b is the number of FN observations, c is the number of FP observations, and d is the number of TN observations.
If either the TPR is 1 or the FPR is 0, then the above equations are undefined. This is avoided by adding 0.5 to the absolute numbers of the TP, FN, TN, and FP observations.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Author contributions: Guarantors of integrity of entire study, P.J.N., J.M.A.v.E.; study concepts, P.J.N., J.M.A.v.E., H.C.W.d.V.; study design, P.J.N., J.M.A.v.E.; definition of intellectual content, P.J.N., J.M.A.v.E., H.C.W.d.V.; literature research, P.J.N., T.L.; data acquisition, P.J.N., T.L.; data analysis, P.J.N., T.L., H.C.W.d.V.; statistical analysis, P.J.N.; manuscript preparation, P.J.N., T.L., J.M.A.v.E.; manuscript editing, P.J.N., T.L., J.M.A.v.E.; manuscript review, T.L., H.C.W.d.V., J.M.A.v.E.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONAPPENDIXREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  R. Met, S. Bipat, D. A. Legemate, J. A. Reekers, and M. J. W. Koelemay Diagnostic Performance of Computed Tomography Angiography in Peripheral Arterial Disease: A Systematic Review and Meta-analysis JAMA, January 28, 2009; 301(4): 415 - 424. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Fenchel, J. Doering, A. Seeger, U. Kramer, K. Rittig, B. Klumpp, C. D. Claussen, and S. Miller Ultrafast Whole-Body MR Angiography with Two-dimensional Parallel Imaging at 3.0 T: Feasibility Study Radiology, January 1, 2009; 250(1): 254 - 263. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Miyazaki and V. S. Lee Nonenhanced MR Angiography Radiology, July 1, 2008; 248(1): 20 - 43. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Thurnher, S. Miller, G. Schneider, C. Ballarati, G. Bongartz, C. U. Herborn, S. Schoenberg, M. A. Cova, G. Morana, K. Niazi, et al. Diagnostic Performance of Gadobenate Dimeglumine Enhanced MR Angiography of the Iliofemoral and Calf Arteries: A Large-Scale Multicenter Trial Am. J. Roentgenol., November 1, 2007; 189(5): 1223 - 1237. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Albrecht, E. Foert, R. Holtkamp, M. A. Kirchin, C. Ribbe, F. K. Wacker, M. Kruschewski, and B. C. Meyer 16-MDCT Angiography of Aortoiliac and Lower Extremity Arteries: Comparison with Digital Subtraction Angiography Am. J. Roentgenol., September 1, 2007; 189(3): 702 - 711. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Fenchel, R. Saleh, H. Dinh, M. H. Lee, K. Nael, M. Krishnam, S. G. Ruehm, S. Miller, J. Child, and J. P. Finn Juvenile and Adult Congenital Heart Disease: Time-resolved 3D Contrast-enhanced MR Angiography Radiology, August 1, 2007; 244(2): 399 - 410. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Cina, R. Ouwendijk, and M. G. M. Hunink First Challenge of MR versus CT Peripheral Angiography: How to Identify a Winner? Radiology, July 1, 2006; 240(1): 302 - 304. [Full Text] [PDF]
|
|
|
|
|
|
P. J. Schaefer, F. P. Boudghene, H. J. Brambs, M. Bret-Zurita, J. L. Caniego, R. A. Coulden, H. B. Gehl, R. Hammerstingl, A. Huber, R. J. Mendez, et al. Abdominal and Iliac Arterial Stenoses: Comparative Double-blinded Randomized Study of Diagnostic Accuracy of 3D MR Angiography with Gadodiamide or Gadopentetate Dimeglumine Radiology, March 1, 2006; 238(3): 827 - 840. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. W.S. Rutjes, J. B. Reitsma, M. Di Nisio, N. Smidt, J. C. van Rijn, and P. M.M. Bossuyt Evidence of bias and variation in diagnostic accuracy studies. Can. Med. Assoc. J., February 14, 2006; 174(4): 469 - 476. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P S Sidhu, P L Allan, F Cattin, D O Cosgrove, A H Davies, D D Do, S Karakagil, J Langholz, D A Legemate, A Martegani, et al. Diagnostic efficacy of SonoVue(R), a second generation contrast agent, in the assessment of extracranial carotid or peripheral arteries using colour and spectral Doppler ultrasound: a multicentre study Br. J. Radiol., January 1, 2006; 79(937): 44 - 51. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Bipat, M. S. van Leeuwen, E. F. I. Comans, M. E. J. Pijl, P. M. M. Bossuyt, A. H. Zwinderman, and J. Stoker Colorectal Liver Metastases: CT, MR Imaging, and PET for Diagnosis--Meta-analysis Radiology, October 1, 2005; 237(1): 123 - 131. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Leiner, A. G. H. Kessels, P. J. Nelemans, G. B. C. Vasbinder, M. W. de Haan, P. E. J. H. M. Kitslaar, K. Y. J. A. M. Ho, J. H. M. Tordoir, and J. M. A. van Engelshoven Peripheral Arterial Disease: Comparison of Color Duplex US and Contrast-enhanced MR Angiography for Diagnosis Radiology, May 1, 2005; 235(2): 699 - 708. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
I. Baumgartner, H. C. Thoeny, O. Kummer, C. Roefke, C. Skjelsvik, C. Boesch, and R. Kreis Leg Ischemia: Assessment with MR Angiography and Spectroscopy Radiology, March 1, 2005; 234(3): 833 - 841. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Bezooijen, H. C. M. van den Bosch, A. V. Tielbeek, G. R. P. Thelissen, K. Visser, M. G. M. Hunink, L. E. M. Duijm, J. Wondergem, J. Buth, and P. W. M. Cuypers Peripheral Arterial Disease: Sensitivity-encoded Multiposition MR Angiography Compared with Intraarterial Angiography and Conventional Multiposition MR Angiography Radiology, April 1, 2004; 231(1): 263 - 271. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M Bossuyt, J. B Reitsma, D. E Bruns, C. A Gatsonis, P. P Glasziou, L. M Irwig, J. G Lijmer, D. Moher, D. Rennie, and H. C. de Vet Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative Fam. Pract., February 1, 2004; 21(1): 4 - 10. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Ota, K. Takase, K. Igarashi, Y. Chiba, K. Haga, H. Saito, and S. Takahashi MDCT Compared with Digital Subtraction Angiography for Assessment of Lower Extremity Arterial Occlusive Disease: Importance of Reviewing Cross-Sectional Images Am. J. Roentgenol., January 1, 2004; 182(1): 201 - 209. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. M. Bossuyt, J. B. Reitsma, D. E. Bruns, C. A. Gatsonis, P. P. Glasziou, L. M. Irwig, J. G. Lijmer, D. Moher, D. Rennie, and H. C. W. de Vet Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative Am. J. Roentgenol., July 1, 2003; 181(1): 51 - 55. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Visser, M. C. J. M. Kock, K. M. Kuntz, M. C. Donaldson, G. S. Gazelle, and M. G. Myriam Hunink Cost-effectiveness Targets for Multi-Detector Row CT Angiography in the Work-up of Patients with Intermittent Claudication Radiology, June 1, 2003; 227(3): 647 - 656. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M. Bossuyt, J. B. Reitsma, D. E. Bruns, C. A. Gatsonis, P. P. Glasziou, L. M. Irwig, J. G. Lijmer, D. Moher, D. Rennie, H. C.W. de Vet, et al. Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative Ann Intern Med, January 7, 2003; 138(1): 40 - 44. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M Bossuyt, J. B Reitsma, D. E Bruns, C. A Gatsonis, P. P Glasziou, L. M Irwig, J. G Lijmer, D. Moher, D. Rennie, and H. C W de Vet Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative BMJ, January 4, 2003; 326(7379): 41 - 44. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M. Bossuyt, J. B. Reitsma, D. E. Bruns, C. A. Gatsonis, P. P. Glasziou, L. M. Irwig, J. G. Lijmer, D. Moher, D. Rennie, and H. C.W. de Vet Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative Clin. Chem., January 1, 2003; 49(1): 1 - 6. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. M. Bossuyt, J. B. Reitsma, D. E. Bruns, C. A Gatsonis, P. P. Glasziou, L. M. Irwig, J. G. Lijmer, D. Moher, D. Rennie, and H. C. W. de Vet Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative Radiology, January 1, 2003; 226(1): 24 - 28. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Loewe, M. Schoder, T. Rand, U. Hoffmann, J. Sailer, T. Kos, J. Lammer, and S. Thurnher Peripheral Vascular Occlusive Disease: Evaluation with Contrast-Enhanced Moving-Bed MR Angiography Versus Digital Subtraction Angiography in 106 Patients Am. J. Roentgenol., October 1, 2002; 179(4): 1013 - 1021. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Z. Huo, M. L. Giger, C. J. Vyborny, and C. E. Metz Breast Cancer: Effectiveness of Computer-aided Diagnosis—Observer Study with Independent Database of Mammograms Radiology, August 1, 2002; 224(2): 560 - 568. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
= observed data points from 2D time-of-flight MR angiography studies, 
= observed data points from 3D gadolinium-enhanced MR angiography studies. The numbers are the reference numbers for the individual studies.
= observed data points from studies in group 2 (ie, studies on 2D time-of-flight MR angiography and image evaluation with MIP plus review of transverse source images or multiplanar reformations), 
= observed data points from studies in group 3 (ie, studies on 3D gadolinium-enhanced MR angiography and image evaluation with only MIP), 
= observed data points from studies in group 4 (ie, studies on 3D gadolinium-enhanced MR angiography and image evaluation with MIP plus review of transverse source images or multiplanar reformations).