Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kinkel, K.
Right arrow Articles by Filly, R. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kinkel, K.
Right arrow Articles by Filly, R. A.
(Radiology. 2000;217:803-811.)
© RSNA, 2000

Genitourinary Imaging

US Characterization of Ovarian Masses: A Meta-Analysis1

Karen Kinkel, MD, Hedvig Hricak, MD, PhD, Ying Lu, PhD, Kyo Tsuda, MD and Roy A. Filly, MD

1 From the Departments of Radiology (K.K., Y.L., H.H., K.T., R.F.) and Biostatistics (Y.L.), University Hospital Geneva, rue Micheli-du-Crest 24, 1211 Geneva 14, Switzerland. Received July 22, 1999; revision requested September 20; final revision received April 13, 2000; accepted May 8. K.K. was supported in part by a grant from the French Radiology Society. Address correspondence to K.K. (e-mail: karen.kinkel@hcuge.ch).


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 
PURPOSE: To compare the effectiveness of current ultrasonographic (US) techniques for characterizing ovarian masses.

MATERIALS AND METHODS: Through a MEDLINE literature search, articles with imaging-histopathologic correlation and data that allowed calculation of contingency tables were identified. Results of morphologic assessment, Doppler US, color Doppler flow imaging, and combined techniques were compared.

RESULTS: Among 89 data sets from 46 included studies (5,159 subjects), 35 sets used morphologic information, 36 measured Doppler US indexes, 10 assessed tumor vascularity with color Doppler flow imaging, and eight used combined techniques. Summary receiver operating characteristic curves revealed significantly higher performance for combined techniques than for morphologic information (P = .003), Doppler US indexes (P = .003), or color Doppler flow imaging alone (P = .001). The Q* point (and 95% CI) for combined techniques was 0.92 (0.87, 0.96) versus 0.85 (0.83, 0.88) for morphology, 0.82 (0.78, 0.86) for Doppler US, and 0.73 (0.58, 0.87) for color Doppler flow imaging. Morphologic assessment showed a trend toward better performance than color Doppler flow imaging (P = .09) or Doppler US indexes (P = .07). Doppler US index results were better in earlier studies (P = .005).

CONCLUSION: Combined US techniques and a diagnostic algorithm perform significantly better than morphologic assessment, color Doppler flow imaging, or Doppler US indexes alone in characterizing ovarian masses.

Index terms: Ovary, neoplasms, 852.31, 852.32 • Ovary, US, 852.12983, 852.12984 • Ultrasound (US), Doppler studies, 852.12983, 852.12984 • Ultrasound (US), comparative studies, 852.12983, 852.12984


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 
Ultrasonography (US) is accepted as the primary imaging modality in the evaluation of an ovarian mass. The use of US in the detection of a suspected ovarian mass and in its differentiation from a uterine mass has been well established. Because US depicts the mass, characterization of the mass is typically performed during the same examination. Thus, de facto, US becomes the main triage method prior to treatment.

A majority of ovarian masses are nonneoplastic cysts. However, when a lesion is suspected of being a neoplasm, surgical intervention must be considered. Twenty-five percent of ovarian neoplasms are malignant (1). For this reason, surgical removal of a suspected ovarian neoplasm is the standard procedure. In most institutions, the type of surgery performed (laparoscopy vs laparotomy) depends on the probability of malignancy. The optimal US technique and diagnostic criteria to use when characterizing a suspected ovarian neoplasm remain controversial. The reported accuracy of US is 65%–94% (2,3) for gray-scale US, 35%–88% (4,5) for color Doppler flow imaging, and 48%–99% (2,6) for Doppler arterial resistance measurements. In addition, diagnostic algorithms and multiparameter scoring systems have been advocated to increase test performance (7–9). The question of which US technique and diagnostic criteria provide the best ovarian lesion characterization has not, to our knowledge, been answered.

Although a meta-analysis does not replace large prospective clinical trials, it has been shown that the results of a meta-analysis in therapeutic trials do not differ from those obtained in large trials (10). Although we do not know whether this result can be applied for trials in which diagnostic tests are evaluated, Irwig et al (11) suggested that a meta-analysis of diagnostic tests represents a potentially powerful tool to summarize the literature by taking into account and analyzing differences between studies. The purpose of this meta-analysis study was to compare the effectiveness of current US techniques in characterizing ovarian masses.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 
Data Sources
A comprehensive literature search of English-language abstracted studies that involved human subjects was performed by using the MEDLINE database and search algorithms, with the keywords "ovarian neoplasm or ovarian cyst or adnexal mass or adnexal lesion" and "ultrasonography or sonography or ultrasound." With consideration of the time of clinical introduction of each modality, searches extended to studies that occurred from December 1985 to December 1998. Only articles in which contingency tables were presented were included. Review articles, letters, comments, and articles without original data were excluded. From a total of 1,536 titles, 121 abstracts were analyzed, and 66 articles were retrieved for data abstraction. Two additional relevant articles were found through manual checking of the reference lists of retrieved articles. A total of 68 articles were found, of which 65 were published in English; one each was published in French, Italian, or German.

Study Selection
The studies included in the meta-analysis met the following inclusion criteria: (a) Patients had an adnexal mass not discovered during screening for ovarian cancer; (b) the reference standard was histopathologic findings; (c) interpreters of US data were blinded to histopathologic findings; (d) presented data allowed calculation of true-positive, true-negative, false-positive, and false-negative imaging results; and (e) data or subsets of data were not published more than once.

Forty-six of 68 references fulfilled the inclusion criteria (Table 1). Reasons for exclusion were absent original data (n = 2), incomplete or inconclusive data (n = 7), absent lesion characterization by the sonologist (n = 8), nonblinded image interpretation (n = 1), lack of histopathologic findings (n = 1), exclusion of borderline tumors (n = 1), and presentation of patients in more than one publication (n = 2).


View this table:
[in this window]
[in a new window]

  		TABLE 1. Results of MEDLINE Search and Study Selection for US Characterization of Ovarian Masses
 
Data Extraction
Two readers (K.K., K.T.) abstracted the data from each article. When disagreement occurred, a third reader reviewed the article and the disagreement was resolved in conference. The readers were not blinded to the origin of publication, journal, or year of publication. The following data were recorded for each article: (a) author and year of publication; (b) sample size; (c) US technique (morphologic assessment, Doppler arterial resistance measurements, color Doppler flow imaging, or combined morphologic, color Doppler flow imaging, and/or Doppler arterial resistance measurements); and (d) congruent true-positive, true-negative, false-positive, and false-negative study results for the identification of a primary, secondary, or borderline ovarian neoplasm versus a benign ovarian neoplasm.

In some studies, only a subgroup of patients fulfilled the inclusion criteria. If an article presented data for more than one US technique, inclusion and exclusion criteria were applied to each technique.

Data Analysis
The meta-analytic method used in our study was based on summary receiver operating characteristic (ROC) curves (12,13). Sensitivity and specificity were recalculated for each reference study by using the conventional corrections for zero counts (14). Because of the lack of independence between sensitivity and specificity (Pearson correlation coefficients of 0.22), the standard method for quantitative integration of data, such as the mean sensitivity and specificity over the studies, was considered inappropriate. To compare US imaging modalities, we used summary ROC analysis, which accounts for the interdependence between sensitivity and specificity. Summary ROC, a mathematic transformation of sensitivity and specificity, has been described by Moses and colleagues (12). The transformed data of all studies were combined through a robust regression (Huber M-regression) analysis (15) in a regression line. The robust regression analysis reduces the effect of heterogeneity among studies by attributing appropriate weights to each study in accordance with its deviation from a normal distribution. The regression line is then transformed back into a summary ROC curve. By combining the data from all studies, summary ROC curves were independent from the diagnostic threshold used to separate benign from malignant ovarian neoplasms.

Following the guidelines for fitting summary ROC curves, we obtained corresponding single number summaries (Q* values). Q* values correspond to the point on the summary ROC curve where sensitivity and specificity are equal. Like the area under the ROC curve, the Q* point indicates how closely a test approaches the desirable performance of 100% sensitivity and specificity. The higher the Q* value, the better the diagnostic test performance. Testing for differences between US techniques was based on Q* values and their associated standard errors.

Covariate Adjustment
To determine whether imaging results were significantly affected by heterogeneity in the studies, we extracted covariates including (a) year of publication; (b) patient characteristics—that is, percentages of pre- and postmenopausal women; percentage of mucinous tumors, endometriomas, and nonneoplastic cysts; prevalence of malignancy; and stage distribution of ovarian cancer—by using International Federation of Gynecology and Obstetrics (FIGO) staging guidelines; (c) study design—that is, prospective versus retrospective data acquisition and description of diagnostic criteria for image interpretation; (d) technical factors—that is, transabdominal versus endovaginal versus combined transabdominal and endovaginal approach, and frequency of US transducer; (e) geographic origin of the study (eg, United States, Europe, or other countries); and (f) professional specialty of authors who interpreted images (eg, gynecology vs radiology).

Covariate adjustment analysis was performed by applying a series of statistical tests in accordance with Moses and colleagues (12) by using regression analysis. The dependent variable of the regression analysis was the difference of the logit of true- and false-positive rates; the independent variables of the regression analysis were the sum of the logit transforms of true- and false-positive rates, the covariate, and its interaction with the sum by following the appendix of de Vries and colleagues (16). The regression was weighted with the inverse of the variance of dependent variables. Each covariate was analyzed separately in each US technique. A multivariate analysis could not be performed because of the small number of studies with complete data on all covariates. A local regression model (17) was used to demonstrate the effect of a covariate on sensitivity and specificity. A covariate significantly affecting the diagnostic performance was plotted against the reported sensitivities and specificities. A P value of .05 was considered to indicate a significant difference.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 
Literature Search
Forty-six studies with 5,159 patients met the inclusion criteria and presented 89 data sets on different US imaging techniques for ovarian mass characterization. Thirty-four studies with 35 data sets published between 1987 and 1998 used morphologic information alone in 3,377 patients (2,3, 6,7,18–47). Twenty-four studies with 36 data sets published between 1992 and 1998 measured Doppler arterial resistance in 2,712 patients. Measurement consisted of the resistive index in 16 data sets (4,7, 22,23,26,28,30,32,33,35,36,38,48–51), the pulsatility index in 15 data sets (3,4,6,23, 27,29,30,36,38,42,43,46,50–52), and other Doppler US variables from five data sets in three studies (4,33,38). In 10 data sets published between 1990 and 1998 (4,5,36, 39,46,49,50,53–55), results were given for internal tumor vascularity, which was assessed by using color Doppler flow imaging in 1,408 patients. In seven studies with eight data sets (Table 2) published between 1992 and 1998, the technique consisted of the combined use of morphologic information and color Doppler flow imaging (8,38), morphologic information and Doppler arterial resistance measurements (7,9,42), or morphologic information, color Doppler flow imaging, and Doppler arterial resistance measurements (2,39) in a total of 832 patients.


View this table:
[in this window]
[in a new window]

  		TABLE 2. Characteristics and Performance of US Studies Using Combined US Techniques
 
Menopausal status was specified in 26 of 46 studies. There were 1,958 (66.5%) premenopausal women among 2,988 subjects. Among the 37 subresults from using arterial Doppler waveform analysis, 16 included premenopausal patients only if the study had been performed during the first 2 weeks of the menstrual cycle. In nine studies, arterial Doppler waveform analysis was performed independently of the phase of the cycle, and in 10 studies, the phase of the cycle at the time of US examination was not indicated. The 5,159 patients who were included in a published study demonstrated 1,275 cancers in 5,238 lesions. Therefore, the prevalence of malignancy was 24%. In 19 articles, the stage of ovarian cancer was specified in 45% of patients with surgical FIGO stage I or II. The mean percentage of mucinous tumors, obtained from 28 studies, was 6.8% (239 of 3,495). In 35 studies, the number of endometriomas was given, with a mean percentage of 14.5% (613 of 4,227). The mean percentage of nonneoplastic cysts, given in 34 studies, was 14.3% (558 of 3,909).

In the 46 included studies, a combined endovaginal and transabdominal approach was used in 24, an endovaginal approach only was used in 18, an abdominal approach only was used in two, and no approach was specified in two. The frequency of the endovaginal transducer was 5.0 MHz in 24 studies, 6.5 MHz in eight studies, 7.0 MHz in five studies, 7.5 MHz in three studies, 5–7 MHz in one study, and unspecified in another study. The frequency of the abdominal transducer was not specified.

Ten studies were performed in the United States; 24, in Europe; and 12, in other countries (five, in Japan; two, in Israel; two, in Taiwan; one, in Brazil; one, in India; and one, in Singapore). The specialty of the author who interpreted the images was radiology in 10 and gynecology in 36 studies. The type of data acquisition was specified in 20 studies, with 16 prospective and four retrospective studies.

Seventy-four subsets indicated diagnostic criteria. Among the diagnostic criteria used for morphologic US, the score published by Sassone and colleagues (56) was applied in 13 subsets; the score published by DePriest et al (57), in two subsets; the criteria published by Granberg and colleagues (58), in two subsets; and the score published by Benacerraf and colleagues (20), in two subsets. Although other subsets listed morphologic diagnostic criteria without a score, in a majority of studies, criteria similar to those of Sassone and colleagues (56) were used. The score included inner-wall structure and thickness, septal presence and thickness, and echogenicity of the mass (56). A score of 9 or more, per Sassone and colleagues (56), was used as the threshold for malignancy. The diagnostic criteria for color Doppler flow imaging considered "flow detection within a mass" as indicative of malignancy. In Doppler US studies in which the pulsatility index was used, a value inferior to 1.0 was used in 67% of the studies; other threshold values were 0.62–1.50. A large range of values was also observed for threshold values of the resistive index, 0.4–0.8.

US Characterization of Ovarian Masses
Because of the lack of independence between sensitivity and specificity, we used summary ROC analysis to compare US techniques. The Q* points of the different US techniques are presented in Table 3 and correspond with the point on the summary ROC curve where sensitivity and specificity are equal. The comparison at Q* points revealed significantly higher performance for combined US techniques than for morphologic assessment (P = .003), Doppler US indexes (P = .003), or color Doppler flow imaging analysis (P = .001) alone (Fig 1). Morphologic assessment showed a trend toward better performance than did color Doppler flow imaging (P = .09) or Doppler US indexes alone (P = .07). The Doppler US indexes employed to estimate arterial resistance performed similarly.


View this table:
[in this window]
[in a new window]

  		TABLE 3. Q* Values for the Assessment of US Characterization of Ovarian Masses
 


View larger version (25K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1. Summary ROC curves comparing US techniques for the characterization of ovarian masses. Summary ROC curve 1 represents combined US techniques; curve 2, morphologic assessment; curve 3, Doppler US index measurements; and curve 4, tumor vascularity assessed with color Doppler flow imaging. All curves demonstrate the same shape; however, the Q* point (intersection between curve and diagonal, where sensitivity and specificity are equal) of the combined techniques is significantly higher than the Q* point of morphologic information alone (P = .003), Doppler US indexes alone (P = .003), or color Doppler flow imaging alone (P = .001).

 
Table 2 shows the characteristics of the studies included in the combined US techniques. The eight included studies were further subdivided into three groups, depending on the information that was used for lesion characterization. The small number of studies in each group—two for morphologic assessment with color Doppler flow imaging, three for morphologic assessment with Doppler arterial resistance measurements, and three with morphologic assessment, color Doppler flow imaging, and Doppler arterial resistance measurements—did not allow further testing of differences in test performance that were based on summary ROC curves. However, the common point in combination studies in which low values of specificity (40%–52%) were achieved (7,42) was an equal weight between morphology and arterial Doppler waveform analysis. In those studies, an ovarian mass was considered malignant if the threshold of the morphologic score or the threshold of the resistive index was obtained. In combination studies with high specificity (93%–100%) and sensitivity values (88%–97%), the approximate weight for morphology versus color Doppler flow imaging or arterial Doppler waveform information was higher than 50%, with a range of 65%–90% (2, 8,9,38).

Subgroup Analysis
To test the validity of the results, a subgroup analysis that took into account potentially significant covariates was performed (Table 4). The analysis showed significantly higher performance in studies with fewer cases of mucinous tumors (P = .027) and in studies in which diagnostic criteria were described (P = .02). This result was valid for both covariates in the overall group (all US techniques) and in the subgroup of arterial Doppler index measurements. Insufficient sample size might be a reason why the other US techniques did not show a significant relationship between those covariates and the study performance. As shown in Figure 2, sensitivity and specificity decreased with the increase of the percentage of mucinous tumors in the study population. Figure 3 demonstrates that the specificity of 74 data sets that described diagnostic criteria was significantly better, at equal sensitivity, than the specificity of 15 data sets without specified diagnostic criteria. This result was obtained independently of the study technique. A trend toward better results was seen in study populations with a lower prevalence of malignancy (P = .15).


View this table:
[in this window]
[in a new window]

  		TABLE 4. US Characterization of Ovarian Masses
 


View larger version (41K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2a. Graphs show (a) summary ROC curves of US characterization of ovarian masses as a function of the percentage of mucinous tumors and (b) sensitivity and specificity as functions of the percentage of mucinous tumors. (a) The summary ROC curve at the top, which corresponds to 0% of mucinous tumors, has the highest performance, as compared with the curve at the bottom, which corresponds to studies with 21% of mucinous tumors in the study population (P = .027). Curves in between correspond to increments of 1% in the percentage of mucinous tumors. Solid lines = sensitivity, dotted local regression line = specificity. (b) Local regression lines show that both sensitivity (+, solid line) and specificity ({circ}, dotted line) are affected by the percentage of mucinous tumors.

 


View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2b. Graphs show (a) summary ROC curves of US characterization of ovarian masses as a function of the percentage of mucinous tumors and (b) sensitivity and specificity as functions of the percentage of mucinous tumors. (a) The summary ROC curve at the top, which corresponds to 0% of mucinous tumors, has the highest performance, as compared with the curve at the bottom, which corresponds to studies with 21% of mucinous tumors in the study population (P = .027). Curves in between correspond to increments of 1% in the percentage of mucinous tumors. Solid lines = sensitivity, dotted local regression line = specificity. (b) Local regression lines show that both sensitivity (+, solid line) and specificity ({circ}, dotted line) are affected by the percentage of mucinous tumors.

 


View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3. Graph shows summary ROC curves that compare US studies with and without diagnostic criteria for ovarian lesion characterization. The ROC curve for US studies with diagnostic criteria (curved solid line) is more favorable than the ROC curve for studies without diagnostic criteria (dotted line). The difference in performance was significant (P = .02).

 
The results of Doppler arterial resistance measurements varied with the year of publication; better results were demonstrated in earlier studies (P = .005). Figure 4 shows that both sensitivity and specificity decreased in more recent studies. This result was independent of sample size. Menopausal status (the percentage of premenopausal women), percentage of endometriomas and nonneoplastic cysts, prospective or retrospective data acquisition, US approach, frequencies of the transducer (within the range specified), geographic origin of the study, and specialty of the sonologist did not reach significance.



View larger version (36K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4a. Graphs show (a) summary ROC curves and (b) nonparametric regression lines of US studies using Doppler arterial resistance measurements for ovarian lesion characterization as a function of year of publication (P = .005). (a) Summary ROC curve at top corresponds to studies published in 1992 that demonstrated better performance than did studies at the bottom published in 1998; studies in between correspond to increments of 1 year. (b) Regression lines show that sensitivity (+, solid line) and specificity ({circ}, dotted line) decreased over the years (P = .005).

 


View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4b. Graphs show (a) summary ROC curves and (b) nonparametric regression lines of US studies using Doppler arterial resistance measurements for ovarian lesion characterization as a function of year of publication (P = .005). (a) Summary ROC curve at top corresponds to studies published in 1992 that demonstrated better performance than did studies at the bottom published in 1998; studies in between correspond to increments of 1 year. (b) Regression lines show that sensitivity (+, solid line) and specificity ({circ}, dotted line) decreased over the years (P = .005).

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
REFERENCES
 
The preoperative imaging characterization of an ovarian mass directly affects surgical decisions and subspecialty referral and is therefore important for patient treatment. Although gray-scale US remains a mainstay of imaging detection of suspected ovarian masses, persisting controversies surround the optimal US techniques for subsequent characterization of the detected ovarian mass.

Our study results demonstrate the superiority of diagnostic US performance when a combination of morphologic and color Doppler flow imaging information (8,38) or morphologic information, color Doppler flow imaging, and Doppler arterial resistance measurements (2,39) are employed. However, studies in which morphologic findings alone are assessed appear superior to studies in which Doppler arterial resistance measurements are used as the sole diagnostic criteria for malignancy. This was particularly true for studies published after 1995. Our subgroup analysis showed a significant decrease in the reported performance of Doppler arterial resistance measurements in studies conducted from 1992 to 1998. An analysis of individual data sets potentially explains this observation. For example, in the article by Kurjak and Predanic (2), 812 patients were included in the study, but results were reported for only 155 patients. Therefore, they and other investigators may have reported on only patients whose Doppler arterial resistance was measured successfully. This selection bias may have been introduced by the noninclusion of subjects who had unsuccessful Doppler arterial resistance measurements. This possibly led to an initial overestimation of the performance capabilities of Doppler arterial resistance measurements.

Although the menopausal status of a woman influences her pretest probability of ovarian cancer, the analysis of studies in which menopausal status is reported showed that the performance of US with or without color Doppler flow imaging was not significantly influenced by the percentage of premenopausal women. US can therefore be applied in pre- and postmenopausal women who are referred for the characterization of an ovarian mass. The preliminary results of previous studies in which the low sensitivity (27) and specificity (51) of US in pre- versus postmenopausal women are not demonstrated by our meta-analysis. Investigators in two studies (8,9) specifically evaluated the importance of "menopausal status" in diagnostic systems by using logistic regression. The results of neither study demonstrated a contribution of the menopausal status to diagnosis.

The characteristics of a study population are important in assessing the external validity of research results (59). Our subgroup analysis of patient characteristics showed that the percentage of mucinous tumors is an important parameter that explains differences in test performance between studies. Mucinous tumors, a subtype of benign and malignant epithelial tumors, represent the third most common benign neoplasm (1) and account for approximately 20% of all malignant epithelial tumors (60). The typical US appearance of a mucinous cystadenoma is a multilocular cystic mass, with the locules commonly containing liquids of different echogenicity (61). In general, morphologic criteria define lesions with greater than three septations as either indeterminate for malignancy or malignant. Portions of the cyst that are echogenic, if mistaken as solid elements, falsely indicate malignancy in these predominantly benign neoplasms. The absence of color Doppler flow in an echogenic portion helps to confirm the cystic nature of the tumor and avoid a potential false-positive diagnosis. This is a common mechanism by which color Doppler flow imaging can be used to interpret gray-scale morphologic features. Nonetheless, benign mucinous tumors remain difficult to diagnose by using all possible US techniques (42,61).

It is not unexpected that a higher prevalence of malignant neoplasm in the patient cohort adversely affected the performance of US techniques in which morphology alone was used. Historically, US has had its greatest success in use in accurately predicting benign status (18,21). The technique is less accurate when used to predict malignancy. Therefore, a greater prevalence of ovarian cancer in the study cohort would likely have a negative effect on diagnostic efficiency.

Results of the meta-analysis demonstrate that the success of US does not appear to be influenced by the specialty training of the sonologist, such as radiology versus gynecology, but rather by the use of meticulous methodology. Using the combination of gray-scale and color Doppler flow imaging findings in a diagnostic system was superior to using morphologic information or optimized thresholds for Doppler arterial resistance measurements alone in scoring systems. The heterogeneity of diagnostic systems used in the combined US techniques did not allow appropriate statistical comparison. However, if we compare studies with similar sensitivities (7,8,42), those combining morphology and Doppler arterial resistance measurements demonstrate lower specificities (40%–52%) (7,42) than do diagnostic systems using morphology and color Doppler flow imaging (specificity, 93%) (8) (Fig 3).

Diagnostic systems requiring the combination of morphology, color Doppler flow imaging, and Doppler arterial resistance measurements (2,39) may be less feasible and more time-consuming, since they require adequate information from all three techniques. The time spent to obtain adequate Doppler arterial resistance measurements may be a factor that limits their larger use (49,53,62). Evidence suggests that multiple resistance indexes must be obtained in each lesion because of the wide variability in arterial resistances that are measured in different areas of any given lesion (28). The lowest resistance index is then used in lesion analysis.

It is unfortunate that there is no mechanism that allows the sonologist to ascertain that the lowest resistance index has been measured. The morphologic appearance of the mass influences the number of Doppler arterial resistance measurements obtained. If two or three "benign" resistance indexes are recorded in a morphologically benign-appearing mass, then the search is concluded. However, if two or three benign resistance indexes are recorded in a morphologically malignant-appearing mass, then the search is continued. Moreover, to our knowledge, the reproducibility of Doppler arterial resistance measurements in ovarian cysts has not yet been verified. However, studies in which Doppler arterial resistance was measured in larger pelvic vessels (63), such as in the ovarian and uterine artery, showed poor agreement. Indeed, the variability in arterial resistances measured in different areas of an ovarian mass (28) limits the feasibility of conducting a study of the reproducibility of Doppler arterial resistance measurements in ovarian lesions. In a study in which morphology alone was used (57), the interobserver agreement of the characterization of ovarian masses was moderate for frequently used criteria such as wall structure ({kappa} = 0.41) and septal structure ({kappa} = 0.47). Therefore, morphology likely contributes to the variability of US results to a lesser degree than does arterial waveform analysis.

Furthermore, the diagnostic systems requiring the combination of morphology, color Doppler flow imaging, and Doppler arterial resistance measurements (2,39), although their performance appears excellent, do not explain how the scoring system was devised. If the diagnostic criteria were first developed in and then applied in the same patient population, then a better fit would be expected; this method results in an overestimation of the technique’s diagnostic performance. This also may help to explain the decline in efficiency noted in studies in more recent years in which arterial resistance indexes were measured (4,6,38,42,51).

Therefore, optimal ovarian lesion characterization appears to be obtained through the combination of gray-scale US morphology and color Doppler flow imaging information. Such a strategy is described in two studies. The system proposed by Buy et al (38) has the advantage of being verified in a large prospective study and does not require calculation. The scoring system in an article by Brown et al (8), obtained through logistic regression, requires the assessment of four parameters: a solid component, the location of color flow in the lesion, the amount of free intraperitoneal fluid, and the presence of septations, combined through a simple addition of subscore values. The presence of a nonhyperechoic solid portion of an ovarian neoplasm, a gray-scale US morphologic feature, had the greatest influence on the diagnosis. Although the scoring system in the article by Brown et al (8) demonstrated a good compromise between sensitivity and specificity, in our opinion, the system needs validation in a prospective study.

The characterization of an ovarian mass as benign or malignant can be achieved by using a list of diagnostic criteria that attribute equal weight to each criterion or by using a scoring system that attributes numbers of increasing value (weights) to each criterion, the sum of which results in a score. If the score of an ovarian mass reaches the previously established threshold for malignancy, the mass is considered malignant. The threshold value for a scoring system should be obtained by performing ROC curve analysis in a large study population that is representative of the target population. When multiple and sometimes conflicting diagnostic features are available, it is reasonable to assume that optimized weights attributed to each diagnostic criterion or the combination of criteria into a score have a greater chance to achieve a correct and reproducible diagnosis than do simple diagnostic rules that require experience. Scoring systems can be obtained through multilogistic regression and ROC curves or artificial neural networks that use a nonlinear regression model (64).

Because of the small number of studies that used similar diagnostic criteria, this meta-analysis could not answer the question regarding which criteria worked best, as compared with others. However, results of individual studies suggest that the inclusion of tumor size proposed by DePriest et al (57) did not improve the results obtained by using the score (31) of Sassone and colleagues. Logistic regression analysis in the combined study by Brown et al (8) demonstrated that an abnormal amount of ascites is indicative of malignancy. Although logistic regression analysis suggested an optimal threshold value close to 0.75 for the resistive and pulsatility index, in a multivariate approach, the incremental value of arterial Doppler waveform analysis compared with that of color Doppler flow imaging, and morphology was not powerful enough to be part of the variables that best predicted malignancy (8).

In conclusion, the results of this meta-analysis provides scientific evidence that US techniques that combine gray-scale US morphologic assessment with tumor vascularity imaging information (color Doppler flow imaging) in a diagnostic system are significantly better in ovarian lesion characterization than Doppler arterial resistance measurements, color Doppler flow imaging, or gray-scale US morphologic information alone. The specialty training of the sonologist does not influence results, provided that the sonologist has used meticulous methodology. Furthermore, specific diagnostic criteria for each US technique must be applied. The patient’s menopausal status does not influence results, but the character of the lesion may; mucinous cystadenomas in particular are difficult to accurately characterize by using any US technique or combination thereof.


   FOOTNOTES
 
Abbreviations: FIGO = International Federation of Gynecology and Obstetrics, ROC = receiver operating characteristic

Author contributions: Guarantors of integrity of entire study, K.K., Y.L., H.H.; study concepts, K.K., H.H., R.A.F.; study design, K.K.; definition of intellectual content, K.K., H.H., R.A.F.; literature research, K.K., K.T.; data acquisition, K.K., K.T.; data analysis, K.K., Y.L.; statistical analysis, Y.L.; manuscript preparation, K.K.; manuscript editing, R.A.F., H.H.; manuscript review, K.K., H.H., Y.L., R.A.F.


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 

  1. Koonings PP, Campbell K, Mishell DR, Jr, Grimes DA. Relative frequency of primary ovarian neoplasms: a 10-year review. Obstet Gynecol 1989; 74:921-926.[Medline]
  2. Kurjak A, Predanic M. New scoring system for prediction of ovarian malignancy based on transvaginal color Doppler sonography. J Ultrasound Med 1992; 11:631-638.[Abstract]
  3. Salem S, White LM, Lai J. Doppler sonography of adnexal masses: the predictive value of the pulsatility index in benign and malignant disease. AJR Am J Roentgenol 1994; 163:1147-1150.[Abstract/Free Full Text]
  4. Tailor A, Jurkovic D, Bourne TH, Natucci M, Collins WP, Campbell S. A comparison of intratumoural indices of blood flow velocity and impedance for the diagnosis of ovarian cancer. Ultrasound Med Biol 1996; 22:837-843.[Medline]
  5. Juhasz B, Kurjak A, Lampe L, Zalud I, Crvenkovic G, Hernadi Z. Tissue characterization by transvaginal colour Doppler for the evaluation of gynaecological tumours. 2. Clinical experiences. Acta Med Hung 1990; 47:149-156.[Medline]
  6. Valentin L. Gray scale sonography, subjective evaluation of the color Doppler image and measurement of blood flow velocity for distinguishing benign and malignant tumors of suspected adnexal origin. Eur J Obstet Gynecol Reprod Biol 1997; 72:63-72.[Medline]
  7. Bromley B, Goodman H, Benacerraf BR. Comparison between sonographic morphology and Doppler waveform for the diagnosis of ovarian malignancy. Obstet Gynecol 1994; 83:434-437.[Medline]
  8. Brown DL, Doubilet PM, Miller FH, et al. Benign and malignant ovarian masses: selection of the most discriminating gray-scale and Doppler sonographic features. Radiology 1998; 208:103-110.[Abstract/Free Full Text]
  9. Alcazar JL, Jurado M. Using a logistic model to predict malignancy of adnexal masses based on menopausal status, ultrasound morphology, and color Doppler findings. Gynecol Oncol 1998; 69:146-150.[Medline]
  10. Cappelleri JC, Ioannidis JP, Schmid CH, et al. Large trials vs meta-analysis of smaller trials: how do their results compare?. JAMA 1996; 276:1332-1338.[Abstract/Free Full Text]
  11. Irwig L, Tosteson AN, Gatsonis C, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 1994; 120:667-676.[Abstract/Free Full Text]
  12. Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med 1993; 12:1293-1316.[Medline]
  13. Midgette AS, Stukel TA, Littenberg B. A meta-analytic method for summarizing diagnostic test performances: receiver-operating-characteristic-summary point estimates. Med Decis Making 1993; 13:253-257.
  14. Haldane JBS. The estimation and significance of the logarithm of a ratio of frequencies. Ann Hum Genet 1955; 20:309-314.
  15. Venables WN, Ripley BD. Robust statistics. Modern applied statistics with S-Plus 2nd ed. New York, NY: Springer, 1997; 247-266.
  16. De Vries SO, Hunink MG, Polak JF. Summary receiver operating characteristic curves as a technique for meta-analysis of the diagnostic performance of duplex ultrasonography in peripheral arterial disease. Acad Radiol 1996; 3:361-369.[Medline]
  17. Chambers JM, Hasties TJ. Statistical models in S Pacific Grove, Calif: Wadsworth & Brooks/Cole, 1991; 309-379.
  18. Herrmann UJ, Jr, Locher GW, Goldhirsch A. Sonographic patterns of ovarian tumors: prediction of malignancy. Obstet Gynecol 1987; 69:777-781.[Medline]
  19. Gadducci A, Capriello P, Bartolini T, et al. The association of ultrasonography and CA-125 test in the preoperative evaluation of ovarian carcinoma. Eur J Gynaecol Oncol 1988; 9:373-376.[Medline]
  20. Benacerraf BR, Finkler NJ, Wojciechowski C, Knapp RC. Sonographic accuracy in the diagnosis of ovarian masses. J Reprod Med 1990; 35:491-495.[Medline]
  21. Granberg S, Norstrom A, Wikland M. Comparison of endovaginal ultrasound and cytological evaluation of cystic ovarian tumors. J Ultrasound Med 1991; 10:9-14.[Abstract]
  22. Schneider VL, Schneider A, Reed KL, Hatch KD. Comparison of Doppler with two-dimensional sonography and CA 125 for prediction of malignancy of pelvic masses. Obstet Gynecol 1993; 81:983-988.[Medline]
  23. Timor-Tritsch LE, Lerner JP, Monteagudo A, Santos R. Transvaginal ultrasonographic characterization of ovarian masses by means of color flow-directed Doppler measurements and a morphologic scoring system. Am J Obstet Gynecol 1993; 168:909-913.[Medline]
  24. Buist MR, Golding RP, Burger CW, et al. Comparative evaluation of diagnostic methods in ovarian carcinoma with emphasis on CT and MRI. Gynecol Oncol 1994; 52:191-198.[Medline]
  25. DePriest PD, Varner E, Powell J, et al. The efficacy of a sonographic morphology index in identifying ovarian cancer: a multi-institutional investigation. Gynecol Oncol 1994; 55:174-178.[Medline]
  26. Jain KA. Prospective evaluation of adnexal masses with endovaginal gray-scale and duplex and color Doppler US: correlation with pathologic findings. Radiology 1994; 191:63-67.[Abstract/Free Full Text]
  27. Kawai M, Kikkawa F, Ishikawa H, et al. Differential diagnosis of ovarian tumors by transvaginal color-pulse Doppler sonography. Gynecol Oncol 1994; 54:209-214.[Medline]
  28. Levine D, Feldstein VA, Babcook CJ, Filly RA. Sonography of ovarian masses: poor sensitivity of resistive index for identifying malignant lesions. AJR Am J Roentgenol 1994; 162:1355-1359.[Abstract/Free Full Text]
  29. Sengoku K, Satoh T, Saitoh S, Abe M, Ishikawa M. Evaluation of transvaginal color Doppler sonography, transvaginal sonography and CA 125 for prediction of ovarian malignancy. Int J Gynaecol Obstet 1994; 46:39-43.[Medline]
  30. Zanetta G, Vergani P, Lissoni A. Color Doppler ultrasound in the preoperative assessment of adnexal masses. Acta Obstet Gynecol Scand 1994; 73:637-641.[Medline]
  31. Botta G, Zarcone R. Trans-vaginal ultrasound examination of ovarian masses in premenopausal women. Eur J Obstet Gynecol Reprod Biol 1995; 62:37-41.[Medline]
  32. Franchi M, Beretta P, Ghezzi F, Zanaboni F, Goddi A, Salvatore S. Diagnosis of pelvic masses with transabdominal color Doppler, CA 125 and ultrasonography. Acta Obstet Gynecol Scand 1995; 74:734-739.[Medline]
  33. Hata K, Hata T, Kitao M. Intratumoral peak systolic velocity as a new possible predictor for detection of adnexal malignancy. Am J Obstet Gynecol 1995; 172:1496-1500.[Medline]
  34. Medl M, Kulenkampff KJ, Stiskal M, Peters-Engl C, Leodolter S, Czembirek H. Magnetic resonance imaging in the preoperative evaluation of suspected ovarian masses. Anticancer Res 1995; 15:1123-1125.[Medline]
  35. Sawicki W, Spiewankiewicz B, Cendrowski K, Stelmachow J. Transvaginal Doppler ultrasound with colour flow imaging in benign and malignant ovarian lesions. Clin Exp Obstet Gynecol 1995; 22:137-142.[Medline]
  36. Stein SM, Laifer-Narin S, Johnson MB, et al. Differentiation of benign and malignant adnexal masses: relative value of gray-scale, color Doppler, and spectral Doppler sonography. AJR Am J Roentgenol 1995; 164:381-386.[Abstract/Free Full Text]
  37. Yamashita Y, Torashima M, Hatanaka Y, et al. Adnexal masses: accuracy of characterization with transvaginal US and precontrast and postcontrast MR imaging. Radiology 1995; 194:557-565.[Abstract/Free Full Text]
  38. Buy JN, Ghossain MA, Hugol D, et al. Characterization of adnexal masses: combination of color Doppler and conventional sonography compared with spectral Doppler analysis alone and conventional sonography alone. AJR Am J Roentgenol 1996; 166:385-393.[Abstract/Free Full Text]
  39. Caruso A, Caforio L, Testa AC, Ciampelli M, Panici PB, Mancuso S. Transvaginal color Doppler ultrasonography in the presurgical characterization of adnexal masses. Gynecol Oncol 1996; 63:184-191.[Medline]
  40. Komatsu T, Konishi I, Mandai M, et al. Adnexal masses: transvaginal US and gadolinium-enhanced MR imaging assessment of intratumoral structure. Radiology 1996; 198:109-115.[Abstract/Free Full Text]
  41. Matthes AC, Moreira de Andrade JM, Bighetti S. Selection of criteria for the treatment of ovarian cysts on the bases of ultrasound and cytology. Gynecol Obstet Invest 1996; 42:244-248.[Medline]
  42. Rehn M, Lohmann K, Rempen A. Transvaginal ultrasonography of pelvic masses: evaluation of B-mode technique and Doppler ultrasonography. Am J Obstet Gynecol 1996; 175:97-104.[Medline]
  43. Strigini FA, Gadducci A, Del Bravo B, Ferdeghini M, Genazzani AR. Differential diagnosis of adnexal masses with transvaginal sonography, color flow imaging, and serum CA 125 assay in pre- and postmenopausal women. Gynecol Oncol 1996; 61:68-72.[Medline]
  44. Tingulstad S, Hagen B, Skjeldestad FE, et al. Evaluation of a risk of malignancy index based on serum CA125, ultrasound findings and menopausal status in the pre-operative diagnosis of pelvic masses. Br J Obstet Gynaecol 1996; 103:826-831.[Medline]
  45. Guerriero S, Mallarini G, Ajossa S, et al. Transvaginal ultrasound and computed tomography combined with clinical parameters and CA-125 determinations in the differential diagnosis of persistent ovarian cysts in premenopausal women. Ultrasound Obstet Gynecol 1997; 9:339-343.[Medline]
  46. Buckshee K, Temsu I, Bhatla N, Deka D. Pelvic examination, transvaginal ultrasound and transvaginal color Doppler sonography as predictors of ovarian cancer. Int J Gynaecol Obstet 1998; 61:51-57.[Medline]
  47. Reuter M, Steffens J, Schuppler U, et al. Critical evaluation of the specificity of MRI and TVUS for differentiation of malignant from benign adnexal lesions. Eur Radiol 1998; 8:39-44.[Medline]
  48. Chou CY, Chang CH, Yao BL, Kuo HC. Color Doppler ultrasonography and serum CA 125 in the differentiation of benign and malignant ovarian tumors. J Clin Ultrasound 1994; 22:491-496.[Medline]
  49. Wu CC, Lee CN, Chen TM, Lai JI, Hsieh CY, Hsieh FJ. Factors contributing to the accuracy in diagnosing ovarian malignancy by color Doppler ultrasound. Obstet Gynecol 1994; 84:605-608.[Medline]
  50. Brown DL, Frates MC, Laing FC, et al. Ovarian masses: can benign and malignant lesions be differentiated with color and pulsed Doppler US?. Radiology 1994; 190:333-336.[Abstract/Free Full Text]
  51. Prompeler HJ, Madjar H, Sauerbrei W. Classification of adnexal tumors by transvaginal color Doppler. Gynecol Oncol 1996; 61:354-363.[Medline]
  52. Weiner Z, Thaler I, Beck D, Rottem S, Deutsch M, Brandes JM. Differentiating malignant from benign ovarian tumors with transvaginal color flow imaging. Obstet Gynecol 1992; 79:159-162.[Medline]
  53. Antonic J, Rakar S. Colour and pulsed Doppler US and tumour marker CA 125 in differentiation between benign and malignant ovarian masses. Anticancer Res 1995; 15:1527-1532.[Medline]
  54. Tepper R, Lerner-Geva L, Altaras MM, et al. Transvaginal color flow imaging in the diagnosis of ovarian tumors. J Ultrasound Med 1995; 14:731-734.[Abstract]
  55. Anandakumar C, Chew S, Wong YC, Chia D, Ratnam SS. Role of transvaginal ultrasound color flow imaging and Doppler waveform analysis in differentiating between benign and malignant ovarian tumors. Ultrasound Obstet Gynecol 1996; 7:280-284.[Medline]
  56. Sassone AM, Timor-Tritsch IE, Artner A, Westhoff C, Warren WB. Transvaginal sonographic characterization of ovarian disease: evaluation of a new scoring system to predict ovarian malignancy. Obstet Gynecol 1991; 78:70-76.[Medline]
  57. DePriest PD, Shenson D, Fried A, et al. A morphology index based on sonographic findings in ovarian cancer. Gynecol Oncol 1993; 51:7-11.[Medline]
  58. Granberg S, Wikland M, Jansson I. Macroscopic characterization of ovarian tumors and the relation to the histological diagnosis: criteria to be used for ultrasound evaluation. Gynecol Oncol 1989; 35:139-144.[Medline]
  59. Hulley SB, Gove S, Browner WS, Cummings SR. Choosing the study subjects: specification and sampling. In: Hulley SB, Cummings SR, eds. Designing clinical research: an epidemiologic approach. Baltimore, Md: Williams & Wilkins, 1988; 18-30.
  60. Katsube Y, Berg JW, Silverberg SG. Epidemiologic pathology of ovarian tumors: a histopathologic review of primary ovarian neoplasms diagnosed in the Denver Standard Metropolitan Statistical Area, 1 July–31 December 1969 and 1 July–31 December 1979. Int J Gynecol Pathol 1982; 1:3-16.[Medline]
  61. Buy JN, Ghossain MA, Sciot C, et al. Epithelial tumors of the ovary: CT findings and correlation with US. Radiology 1991; 178:811-818.[Abstract/Free Full Text]
  62. Tekay A, Jouppila P. Controversies in assessment of ovarian tumors with transvaginal color Doppler ultrasound. Acta Obstet Gynecol Scand 1996; 75:316-329.[Medline]
  63. Farquhar CM, Rae T, Thomas DC, Wadsworth J, Beard RW. Doppler ultrasound in the nonpregnant pelvis. J Ultrasound Med 1989; 1989:451-457.
  64. Biagiotti SR, Desii C, Vanzi E, Gacci G. Predicting ovarian malignancy: application of artificial neural networks to transvaginal and color Doppler flow US. Radiology 1999; 210:399-403.[Abstract/Free Full Text]



This article has been cited by other articles:


Home page
 J Ultrasound MedHome page
P. Yoruk, O. Dundar, B. Yildizhan, L. Tutuncu, and T. Pekin
Comparison of the Risk of Malignancy Index and Self-Constructed Logistic Regression Models in Preoperative Evaluation of Adnexal Masses
J. Ultrasound Med., October 1, 2008; 27(10): 1469 - 1477.
[Abstract] [Full Text] [PDF]

Home page
 J Ultrasound MedHome page
A. Barua, J. S. Abramowicz, J. M. Bahr, P. Bitterman, A. Dirks, K. A. Holub, E. Sheiner, M. J. Bradaric, S. L. Edassery, and J. L. Luborsky
Detection of Ovarian Tumors in Chicken by Sonography: A Step Toward Early Diagnosis in Humans?
J. Ultrasound Med., July 1, 2007; 26(7): 909 - 919.
[Abstract] [Full Text] [PDF]

Home page
 Hum ReprodHome page
J. L. Alcazar, G. Castillo, M. Jurado, and G. L. Garcia
Is expectant management of sonographically benign adnexal cysts an option in selected asymptomatic premenopausal women?
Hum. Reprod., November 1, 2005; 20(11): 3231 - 3234.
[Abstract] [Full Text] [PDF]

Home page
 RadiologyHome page
K. Kinkel, Y. Lu, A. Mehdizade, M.-F. Pelte, and H. Hricak
Indeterminate Ovarian Mass at US: Incremental Value of Second Imaging Test for Characterization--Meta-Analysis and Bayesian Analysis
Radiology, July 1, 2005; 236(1): 85 - 94.
[Abstract] [Full Text] [PDF]

Home page
 J Ultrasound MedHome page
L. Clarke, A. Edwards, and K. Pollard
Acoustic Streaming in Ovarian Cysts
J. Ultrasound Med., May 1, 2005; 24(5): 617 - 621.
[Abstract] [Full Text] [PDF]

Home page
 J Ultrasound MedHome page
H. Marret, S. Sauget, B. Giraudeau, M. Brewer, J. Ranger-Moore, G. Body, and F. Tranquart
Contrast-Enhanced Sonography Helps in Discrimination of Benign From Malignant Adnexal Masses
J. Ultrasound Med., December 1, 2004; 23(12): 1629 - 1639.
[Abstract] [Full Text] [PDF]

Home page
 Hum ReprodHome page
C.D. de Kroon, H.A.G.M. van der Sandt, J.C. van Houwelingen, and F.W. Jansen
Sonographic assessment of non-malignant ovarian cysts: does sonohistology exist?
Hum. Reprod., September 1, 2004; 19(9): 2138 - 2143.
[Abstract] [Full Text] [PDF]

Home page
 RadioGraphicsHome page
P. J. Woodward, K. Hosseinzadeh, and J. S. Saenger
From the Archives of the AFIP: Radiologic Staging of Ovarian Carcinoma with Pathologic Correlation
RadioGraphics, January 1, 2004; 24(1): 225 - 246.
[Abstract] [Full Text] [PDF]

Home page
 RadiologyHome page
M.-R. Orden, J. S. Jurvelin, and P. P. Kirkinen
Kinetics of a US Contrast Agent in Benign and Malignant Adnexal Tumors
Radiology, February 1, 2003; 226(2): 405 - 410.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kinkel, K.
Right arrow Articles by Filly, R. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kinkel, K.
Right arrow Articles by Filly, R. A.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE