(Radiology. 2000;217:516-520.)
© RSNA, 2000
Measurement of Endometrial Thickness at US in Multicenter Drug Trials: Value of Central Quality Assurance Reading1
Miriam A. Bredella, MD,
Vickie A. Feldstein, MD,
Roy A. Filly, MD,
Ruth B. Goldstein, MD,
Peter W. Callen, MD and
Harry K. Genant, MD
1 From the Department of Radiology, University of California, San Francisco, 505 Parnassus Ave, Suite M-392, San Francisco, CA 94143-0628. From the 1999 RSNA scientific assembly. Received November 15, 1999; revision requested January 7, 2000; revision received March 20; accepted April 4. Address correspondence to M.A.B. (e-mail: mbredella@earthlink.net).
 |
ABSTRACT
|
|---|
PURPOSE: To assess the value of central quality assurance (QA) reading of transvaginal ultrasonographic (US) images obtained to measure endometrial thickness and to assess image quality.
MATERIALS AND METHODS: Results of 2,000 US examinations performed in 1,000 subjects during one of two multicenter drug trials were evaluated. Endometrial thickness was measured at the study site; images were then sent to the QA center, where an experienced sonologist evaluated endometrial thickness and image quality.
RESULTS: In 360 (18%) of the 2,000 examinations, image quality was insufficient for central QA reading. Repeat examinations were requested, and suggestions for improvement in technique were provided. In 349 (97%) of the 360 examinations, repeat US images were of acceptable quality. In 99 (5%) of the 1,989 examinations in which endometrial thickness was measured, central measurement of thickness differed by more than 2 mm from that of the site. In a group (n = 300) that was followed up for 1 year, requests for repeat US examinations decreased from 24% at baseline to 11% at 1 year.
CONCLUSION: Central QA reading provides a consistent evaluation of endometrial thickness on US images obtained in multicenter drug trials and helps to ensure the acquisition of high-quality transvaginal US images. It further leads to demonstrable improvement in site performance.
Index terms: Ultrasound (US), quality assurance • Ultrasound (US), transvaginal, 854.1298 • Uterus, endometrium, 854.91
|
INTRODUCTION
|
|---|
It has been shown that careful monitoring and quality assurance (QA) of patient data are needed to ensure the safety of participants in clinical trials. For long-term trials with many participants, it is becoming a standard practice to establish a group of experienced readers, independent of the trial investigators, to perform this monitoring task (1–6). In multicenter pharmaceutical trials in which hormone replacement therapy or selective estrogen receptor modulators (which are known or suspected to have effects on the endometrium) are investigated, the accurate assessment of endometrial changes has become an important safety feature. Changes in endometrial thickness can be associated with endometrial polyps, hyperplasia, or carcinoma. Therefore, the effect on the endometrium might be the critical feature in the selection of an investigational drug (2,7–9).
Transvaginal ultrasonography (US) has been shown to be an excellent method for evaluating the endometrium and has become the method of choice in assessing uterine status in clinical trials (2,7,10–12). Accurate assessment of endometrial thickness becomes difficult when several clinical sites are involved in a trial, due in part to the range of US systems available and the fact that US examinations are operator dependent. Therefore, in multicenter trials, QA and central assessment of transvaginal US findings by experienced readers should provide consistency and standardization in the acquisition of images and in the measurement of endometrial thickness among participating centers because feedback is given to the clinical sites.
Experienced readers are also helpful in establishing standardized study protocols, in writing training manuals for the acquisition of transvaginal US images, and in training site representatives prior to the onset of the clinical trial.
The purpose of this study was to assess the value of central QA reading of transvaginal US images obtained for the measurement of endometrial thickness and to assess the quality of images obtained in multicenter drug trials.
|
MATERIALS AND METHODS
|
|---|
A total of 2,000 transvaginal US examinations were performed in 1,000 postmenopausal women who participated in one of two multicenter clinical drug trials. The trials were conducted to investigate hormone replacement therapy protocols and selective estrogen receptor modulators. Thirty clinical sites participated in the two studies. The trials were approved by the institutional review board at each site, and informed consent was obtained from each study participant prior to enrollment in the study.
Before the beginning of the study, centralized training in the acquisition of transvaginal US images was provided at the QA center. The principal investigator and sonographer responsible for performing transvaginal US attended the training sessions. The images were acquired at the clinical site where the local reader (either a radiologist or a gynecologist) measured endometrial thickness. The study manual for the acquisition of transvaginal US images was created and distributed by staff at the QA center.
Transvaginal US was performed by using a 5.0–7.5-MHZ endovaginal transducer. The uterus was scanned in the sagittal plane from the lateral aspect on one side to the lateral aspect on the other side. The endometrial stripe was measured at its maximum anteroposterior thickness along the longitudinal axis of the uterine body. This measurement included both anterior and posterior endometrial layers. It was obtained by placing electronic calipers at the anterior and posterior uterine walls at the margins of the basal layers of the endometrium delineated by the echogenic interface between endometrium and inner myometrium (Fig 1). If the endometrium was focally thickened, the calipers were placed at the site of the maximal anteroposterior diameter (Fig 2). In the presence of intrauterine fluid, the thicknesses of the anterior and posterior endometrial layers were measured separately and summed (Fig 3).
To ensure consistent and correct reading of endometrial thickness at all centers, the original transvaginal US images were sent to the QA center for confirmation. The role of the QA center staff was to set the acquisition protocol, ensure compliance, evaluate the quality of the transvaginal US images, and measure endometrial thickness. Central QA reading was performed by one of four experienced (7–25 years of experience) sonologists (V.A.F., R.A.F., R.B.G., P.W.C.) who evaluated the quality of the transvaginal US images and determined endometrial thickness.
At least five images were requested per examination, and between one and eight transvaginal US images per examination were received. If fewer than five images were received and, regardless of the number of images received, if the endometrium was not adequately demonstrated, the images were returned to the site with a request to obtain further images.
The transvaginal US images were evaluated for appropriate gain and depth settings, adequate demonstration of the endometrial stripe, and presence of myometrial disease. The sonologists also evaluated the images for correct caliper placement and measured endometrial thickness. The endometrial thickness measurement determined at the site was compared with that of the central reader, and discrepancies were documented. No repeat examination was requested if there was a discrepancy in measurement when image quality was deemed adequate. However, the site was informed if the difference in measurement was more than 2 mm.
The cutoff point of 2 mm was chosen, in part, on the basis of findings that have shown that the interobserver variation in the measurement of endometrial thickness among experienced readers is 2 mm or less (13,14). If the quality of the images was inadequate, the central reader made suggestions for improvement and requested a repeat examination. No endometrial thickness measurement was obtained on images of suboptimal quality. If a repeat examination was requested, or if the difference in measurement resulted in altered patient care, double reading was performed at the central QA center. In some instances, on-site training and experienced consultation were provided to a participating site to optimize image acquisition.
One group of subjects (n = 300) was followed up for 1 year. Images obtained in this group were reviewed to determine if site performance indicators (image quality and measurement accuracy) improved over time on the basis of suggestions provided by the central reader. The 300 subjects underwent between one and six transvaginal US examinations (N = 1,000) during that year.
|
RESULTS
|
|---|
We performed a total of 2,000 transvaginal US examinations in 1,000 subjects in 1 year. One thousand transvaginal US examinations were performed at the beginning of the clinical study, and 1,000 were performed during the year in which the study group of 300 women was followed up. Of these 2,000, 1,640 (82%) examinations resulted in images of sufficient quality for central QA reading (Fig 4). In 360 (18%) of the 2,000 examinations, the quality of the images sent to the QA center was deemed inadequate for central QA reading. After reviewing the submitted images, the reader at the QA center made suggestions for improvement and requested a repeat examination. Initial and requested repeat transvaginal US were counted as one examination.
The most common reasons given for an inadequate transvaginal US image were related to the US technique. Of the 360 rejected sets of examination findings, 144 (40%) were acquired by using inappropriate gain (brightness) settings. The majority of these images were too dark (Fig 5). In 122 (34%) of the examinations, the depth setting was incorrect. On these images, the field of view was inappropriately large such that the uterus was in the near field and did not fill the frame (Fig 6a). In 58 (16%) of the examinations, the endometrial stripe was not adequately visualized (Fig 7a), and a specific cause was not identified. With 36 (10%) of the examinations, subject factors (such as the presence of uterine leiomyomas) interfered with image acquisition.
View larger version (132K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5. Sagittal transvaginal US image that was deemed insufficient for central QA reading. Repeat examination was requested and resulted in the acquisition of a good-quality image. To improve image quality, an increase in gain (brightness) was suggested.
|
|
View larger version (200K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 6a. (a) Sagittal transvaginal US image determined to be of insufficient quality by the reader at the QA center. The site was instructed to adjust the depth and/or scale settings so that the uterus filled the screen. On an image such as this obtained with a suboptimal depth setting, a minor error in the placement of the calipers may result in substantial measurement error. (b) Sagittal transvaginal US image obtained at repeat examination shows that the uterus appropriately fills the screen and that the calipers are placed adequately. (Inner calipers were used to measure the endometrial stripe, and outer calipers were used to measure the uterus).
|
|
View larger version (228K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 6b. (a) Sagittal transvaginal US image determined to be of insufficient quality by the reader at the QA center. The site was instructed to adjust the depth and/or scale settings so that the uterus filled the screen. On an image such as this obtained with a suboptimal depth setting, a minor error in the placement of the calipers may result in substantial measurement error. (b) Sagittal transvaginal US image obtained at repeat examination shows that the uterus appropriately fills the screen and that the calipers are placed adequately. (Inner calipers were used to measure the endometrial stripe, and outer calipers were used to measure the uterus).
|
|
View larger version (171K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 7a. (a) Transverse transvaginal US image was deemed insufficient for central reading because the endometrial stripe is not adequately shown. Placement of the calipers was not evaluated on this image of suboptimal quality. (b) At repeat examination, the technique was improved, and the endometrial stripe (arrows) is well depicted on this transverse image.
|
|
View larger version (176K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 7b. (a) Transverse transvaginal US image was deemed insufficient for central reading because the endometrial stripe is not adequately shown. Placement of the calipers was not evaluated on this image of suboptimal quality. (b) At repeat examination, the technique was improved, and the endometrial stripe (arrows) is well depicted on this transverse image.
|
|
With 349 (97%) of the 360 sets of examination findings that were initially rejected, repeat examination resulted in images of acceptable quality (Figs 6b, 7b). With 11 (3%) of the examinations, repeat examination failed to result in images of acceptable quality. In most of these instances, subject factors interfered with adequate image acquisition. In some instances, these subjects underwent hysteroscopy or sonohysterography and continued to participate in the study if results of these procedures were normal.
Overall, endometrial thickness measurements were obtained in 1,989 examinations. In 99 (5%) of the 1,989 examinations, the endometrial thickness measurement provided by the central reader differed from that of site reader by more than 2 mm. The most common reasons for the difference in measurement were incorrect caliper placement at the correct site and measurement at an incorrect location along the endometrial stripe. In 79 (4%) of the 1,989 examinations, the measurement of the QA center was greater than that of the site by more than 2 mm. In 20 (1%) examinations, the measurement of the QA center was less than that of the site by more than 2 mm. In 40 (40%) of the 99 examinations in which endometrial thickness measurements differed by more than 2 mm, the difference in the measurements provided by the central reader and the site resulted in alterations in treatment on the basis of the study protocols (eg, prompted the need for endometrial biopsy or sonohysterography).
One study group of 300 subjects was followed up for 1 year. We performed a total of 1,000 transvaginal US examinations in these 300 subjects (between one and six examinations per subject) over that time. The percentage of requested repeat examinations in this group decreased from 24% at baseline to 11% after 1 year (Table). The percentage of examinations in which the central and local readings of endometrial thickness differed by more than 2 mm decreased from 5% to 3% in this group.
|
DISCUSSION
|
|---|
Centralized data monitoring and QA, as well as standardization of study protocols and training manuals, are important aspects of large multicenter clinical trials (1,5,6,15–17). For monitoring and quality assurance to be productive, they must be performed by experienced readers who have the necessary training and skills. Inadequate QA performance can severely jeopardize group trials. The establishment and implementation of performance standards, monitoring, and timely communication help ensure successful conduct of multicenter clinical trials (1,3,4,15). However, the implementation of QA procedures is costly, and it is necessary to assess whether the methods used are effective and worthwhile.
Findings of previous studies on the use of hormone replacement therapy and tamoxifen citrate have shown the need for evaluating the endometrium and for monitoring endometrial thickness (7,8). The introduction of transvaginal US has improved our ability to visualize and assess changes in endometrial morphology and thickness (10,12,18). The examination is safe and painless, widely available, relatively affordable, and well tolerated by the patient. Therefore, transvaginal US imaging is frequently used for monitoring the thickness and morphology of the endometrium in subjects receiving investigational drugs.
Accurate reading of endometrial thickness is particularly difficult if the quality of the transvaginal US images is suboptimal, as it was in 360 (18%) of the 2,000 submitted sets of examination findings. To accurately assess the effect of investigational drugs on the endometrium, the quality and reliability of the transvaginal US images of the endometrium should be carefully studied to ensure that they are sufficient and that the endometrial thickness measurement is correct. Further, experienced readers who participate in QA reading can provide an important added benefit by offering suggestions for improving the acquisition of transvaginal US images and by providing training at the site.
In our study, repeat examinations in 349 (97%) of the 360 initial examinations judged to be technically inadequate resulted in the acquisition of adequate images. Further, in the subgroup of 300 subjects, the percentage of transvaginal US examinations deemed to be of insufficient quality decreased from 24% to 11% after 1 year. By reducing the number of repeat examinations, the overall costs of the trial and the imposition on research subjects are reduced.
The performance and interpretation of all US findings is operator-dependent. Reproducibility of endometrial thickness measurements is related to the inherent variability in US measurements and in the experience of the examiner. Study findings have shown that the error in the reproducibility of endometrial thickness measurements at transvaginal US between experienced readers can be as high as 2 mm and that the variability of the measurements between experienced and inexperienced observers can be as much as 18 mm (13,14). Incorrect interpretation of endometrial thickness can result in clinically important errors in management: Underestimation (false-negative finding) of endometrial thickness can result in undetected pathologic findings, and overestimation (false-positive finding) can result in unnecessary procedures for the patient.
The potential for errors in measurement can be exacerbated by common technical errors, namely, use of an inadequate depth setting (often a field of view that is too large) and consequent inaccurate caliper placement. Often, at repeat examination with use of an appropriate depth setting, correct caliper placement can be achieved.
During 1-year follow-up in the study group of 300, the percentage of instances in which the endometrial thickness measurement determined at central reading differed by more than 2 mm from that determined at site reading decreased from 5% to 3%. With improved image quality and optimal image acquisition, agreement between the site and central readings also improved.
Our study had a couple of limitations. The first is that the intra- and interobserver variation in endometrial thickness measurement among the four experienced sonologists were not analyzed before this investigation. However, study findings have indicated that the interobserver variation among experienced readers is 2 mm or less (13,14). The second is that we did not have access to the results of other examinations (eg, saline infusion sonohysterography, endometrial biopsy, hysteroscopy). Of particular interest are the results in those patients who, according to the study protocols, underwent an additional procedure due to the differences between the measurements of endometrial thickness provided by the QA center and the site. It is presumed that when differences occur, the measurement of endometrial thickness obtained at the QA center is more accurate than that obtained at the site; however, without pathologic correlation as proof, we realize that we are limited in our assertion.
In conclusion, central QA determination of endometrial thickness and image quality is helpful in multicenter drug trials. Central QA reading helps to ensure the acquisition of high-quality transvaginal US images and provides a consistent evaluation of endometrial thickness measurements. It also leads to demonstrable improvement in site performance.
|
FOOTNOTES
|
|---|
This paper received a Fellow Trainee Prize at the 1999 RSNA Scientific Assembly and Annual Meeting
Abbreviation: QA = quality assurance
Author contributions: Guarantors of integrity of entire study, M.A.B., V.A.F., H.K.G.; study concepts, M.A.B., V.A.F., H.K.G.; study design, M.A.B., V.A.F.; definition of intellectual content, all authors; literature research, M.A.B., V.A.F.; clinical studies, V.A.F., R.A.F., R.B.G., P.W.C.; data acquisition and analysis, M.A.B., V.A.F., R.A.F., R.B.G., P.W.C.; manuscript preparation, M.A.B., V.A.F.; manuscript editing, V.A.F., R.A.F., R.B.G., P.W.C., H.K.G.; manuscript review, V.A.F., H.K.G.
|
REFERENCES
|
|---|
-
Knatterud GL, Rockhold FW, George SL, et al. Guidelines for quality assurance in multicenter trials: a position paper. Control Clin Trials 1998; 19:477-493.[Medline]
-
Fleischer AC, Wheeler JE, Yeh IT, Kravitz B, Jensen C, MacDonald B. Sonographic assessment of the endometrium in osteopenic postmenopausal women treated with idoxifene. J Ultrasound Med 1999; 18:503-512.[Abstract]
-
Nowak R. Problems in clinical trials go far beyond misconduct. Science 1994; 264:1538-1541.[Free Full Text]
-
Cohen J. Clinical trial monitoring: hit or miss?. Science 1994; 264:1534-1537.[Free Full Text]
-
Lu Y, Mathur AK, Blunt BA, et al. Dual x-ray absorptiometry quality control: comparison of visual examination and process-control charts. J Bone Miner Res 1996; 11:626-637.[Medline]
-
Gluer CC, Barkmann R, Heller M. Ultrasound: already clinically relevant?. Ther Umsch 1998; 55:667-675[German].[Medline]
-
Bertelli G, Venturini M, Del Mastro L, et al. Tamoxifen and the endometrium: findings of pelvic ultrasound examination and endometrial biopsy in asymptomatic breast cancer patients. Breast Cancer Res Treat 1998; 47:41-46.[Medline]
-
Cummings SR, Eckert S, Krueger KA, et al. The effect of raloxifene on risk of breast cancer in postmenopausal women: results from the MORE randomized trial—multiple outcomes of raloxifene evaluation. JAMA 1999; 281:2189-2197.[Abstract/Free Full Text]
-
Granberg S, Yleostalo P, Wikland M, Karlsson B. Endometrial sonographic and histologic findings in women with and without hormonal replacement therapy suffering from postmenopausal bleeding. Maturitas 1997; 27:35-40.[Medline]
-
Dijkhuizen FP, Breolmann HA, Potters AE, Bongers MY, Heinz AP. The accuracy of transvaginal ultrasonography in the diagnosis of endometrial abnormalities. Obstet Gynecol 1996; 87:345-349.[Medline]
-
Bourne T, Hamberger L, Hahlin M, Granberg S. Ultrasound in gynecology: endometrium. Int J Gynecol Obstet 1997; 56:115-127.[Medline]
-
Granberg S, Wikland M, Karlsson B, Norstreom A, Friberg LG. Endometrial thickness as measured by endovaginal ultrasonography for identifying endometrial abnormality. Am J Obstet Gynecol 1991; 164:47-52.[Medline]
-
Wolman I, Jaffa AJ, Sagi J, Hartoov J, Amster R, David MP. Transvaginal ultrasonographic measurements of endometrial thickness: a reproducibility study. J Clin Ultrasound 1996; 24:351-354.[Medline]
-
Delisle MF, Villeneuve M, Boulvain M. Measurement of endometrial thickness with transvaginal ultrasonography: is it reproducible?. J Ultrasound Med 1998; 17:481-484.[Abstract]
-
Meinert CL. Clinical trials and treatment effects monitoring. Control Clin Trials 1998; 19:515-522.[Medline]
-
Pollock BH. Quality assurance for interventions in clinical trials: multicenter data monitoring, data management, and analysis. Cancer 1994; 74:2647-2652.[Medline]
-
Wu CY, Li J, Jergas M, Genant HK. Comparison of semiquantitative and quantitative techniques for the assessment of prevalent and incident vertebral fractures. Osteoporos Int 1995; 5:354-370.[Medline]
-
Smith-Bindman R, Kerlikowske K, Feldstein VA, et al. Endovaginal ultrasound to exclude endometrial cancer and other endometrial abnormalities. JAMA 1998; 280:1510-1517.[Abstract/Free Full Text]
TOP
ABSTRACT
INTRODUCTION
