Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Bankier, A. A.
Right arrow Articles by Gevenois, P. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Bankier, A. A.
Right arrow Articles by Gevenois, P. A.
(Radiology. 1999;211:851-858.)
© RSNA, 1999


Thoracic Imaging

Pulmonary Emphysema: Subjective Visual Grading versus Objective Quantification with Macroscopic Morphometry and Thin-Section CT Densitometry1

Alexander A. Bankier, MD, Viviane De Maertelaer, PhD, Caroline Keyzer, MD and Pierre Alain Gevenois, MD, PhD

1 From the Department of Radiology, University of Vienna, Waehringer Guertel 18-20, A-1090 Vienna, Austria (A.A.B.) and the Department of Radiology, Hôpital Erasme (C.K., P.A.G.) and the Statistical Unit, Institute for Interdisciplinary Research in Human Biology and Nuclear Medicine (V.D.M.), Université Libre de Bruxelles, Belgium. Received June 22, 1998; revision requested August 8; revision received September 14; accepted December 9. Address reprint requests to A.A.B.


    Abstract
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
PURPOSE: To compare subjective visual grading of pulmonary emphysema with macroscopic morphometry and computed tomographic (CT) densitometry.

MATERIALS AND METHODS: In 62 consecutive patients who underwent thin-section CT before surgical lung resection, emphysema was objectively quantified with computer-assisted macroscopic morphometry and CT densitometry. The percentage of lung macroscopically occupied by emphysema was compared with the percentage occupied on CT scans by pixels with attenuation values lower than a predefined threshold (CT densitometry). Three readers with varying degrees of expertise subjectively graded emphysema with visual assessment at two reading sessions. Data from objective quantification and subjective grading were analyzed with correlation coefficients, and interobserver and intraobserver agreement were calculated.

RESULTS: Subjective grading of emphysema showed less agreement with the macroscopic reference standard results (r = 0.439–0.505; P < .05) than with objective CT densitometric results (r = 0.555–0.623; P < .001). The 95% CIs for the intercepts of the linear regression lines were suggestive of systematic subjective overestimation of emphysema by all three readers. Interobserver agreement was moderate ({kappa} = 0.431–0.589). Intraobserver agreement was good to excellent ({kappa} = 0.738–0.936). The expertise of individual readers did not substantially influence results.

CONCLUSION: Systematic overestimation and moderate interobserver agreement may compromise subjective visual grading of emphysema, which suggests that subjective visual grading should be supplemented with objective methods to achieve precise, reader-independent quantification of emphysema.

Index terms: Computed tomography, quantitative, 60.12111 • Emphysema, pulmonary, 60.751 • Lung, CT, 60.12111, 60.12118 • Lung, density


    Introduction
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
In the past decade, a number of studies (14) in which subjective computed tomographic (CT) grading of emphysema was used have resulted in statistically significant correlations between visual scores and the panel of standards described by Thurlbeck et al (5). In all these studies, however, both the CT images and the pathologic cut sections from fixed lungs represented axial cross sections, whereas the sections used in the Thurlbeck et al panel of standards were parasagittal. Comparison of these two types of images, therefore, required intuitive adjustment (2,3) or modification of the original panel used as a reference (4). Moreover, the panel of standards was originally designed for Gough-Wentworth paper-mounted lung sections and not for the cut surface of lung specimens.

In addition, the grading system described by Thurlbeck et al (5) and referred to in previous CT studies (13) was not designed to help quantify emphysema but to rank emphysema according to categories of severity (5). As a consequence, correlations between subjective visual CT scores and the Thurlbeck et al panel of standards indicate only that visual CT scores and pathologic results are linked. This, however, does not imply that the percentage areas obtained at visual CT quantification are equal to those obtained at pathologic assessment.

To overcome these limitations, a computer-assisted method was developed for the objective quantification of emphysema (6). This macroscopic method applies to horizontal, paper-mounted lung sections similar to the imaging sections obtained at axial CT. By using this method, it is possible to compare the percentage of lung macroscopically occupied by emphysema with the percentage of lung occupied by pixels below a predefined CT attenuation threshold.

This method, termed CT densitometry, has been validated with macroscopic and microscopic comparisons (7,8); however, objective quantification of emphysema is time-consuming and more expensive than subjective visual grading. Therefore, subjective visual grading of emphysema still is commonly used to correlate CT morphology with functional clinical data (9,10). This, however, assumes that subjective visual grading of emphysema on CT images is as accurate as objective quantitative methods. To test the validity of this assumption, we designed the present study. The aim of this study was to analyze the performance of subjective visual grading of pulmonary emphysema on in vivo CT scans and to compare the results with those of the objective methods of macroscopic morphometry and CT densitometry.


    MATERIALS AND METHODS
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Patients
This study originally included 89 consecutive patients admitted to the Hôpital Erasme (Brussels, Belgium) between October 1991 and September 1992. All patients had been referred to the thoracic surgery department for surgical resection of lung cancer or for lung transplantation because of emphysema. Patients with previous lung surgery (n = 1), microscopic evidence of interstitial lung disease in nontumoral parenchyma (n = 2), pleural disease (n = 1), or pneumonia or parenchymal collapse (n = 9) were excluded from further investigation. We also excluded a patient who underwent segmentectomy. Of the remaining 75 patients, 13 were not surgically treated because of lymph node involvement detected during preoperative mediastinoscopy, and these patients also were excluded. Thus, a final series of 62 patients were included in this study.

There were 10 women and 52 men, with a mean age (±SD) of 62 years ± 9 (age range, 36–77 years). Of this group, 58 patients underwent lobar or whole-lung resection because of cancer, and four underwent lung transplantation. Of the four patients who underwent lung transplantation, one had {alpha}1-antitrypsin deficiency. Microscopic analysis in this patient, a heavy smoker, revealed a predominantly centrilobular, mixed type of emphysema.

This investigation was approved by the hospital ethical committee, and informed consent was obtained from all patients.

CT Examinations
Thin-section CT scans were obtained with a commercially available scanner (Somatom Plus; Siemens Medical Systems, Erlangen, Germany) during a breath hold at full inspiration. All patients were examined in the supine position, and none of the patients received contrast material. The acquisition time was 1 second per section, and the tube current was 137 kVp at 180 mA. Section thickness was 1 mm, with a 10-mm intersection interval. All examinations were performed from the apex to the base of the lungs. Images were reconstructed by using a bone algorithm at a display window width of 1,600 HU and a window center of -600 HU. Images were photographed with twelve images per sheet of 35.5 x 43.0-cm film.

Objective Assessment
Macroscopic assessment.—Immediately after surgery, the resected lobes were inflated with 10% buffered formalin at a distending pressure of 25–30 cm H2O for 12 hours. By using a modified Gough-Wenthworth technique (11,12), a total of 660 horizontal whole-lung sections were obtained in the patient population as a whole; in individual lungs, sections were obtained every 1–2 cm. The mean number of sections from surgical specimens that were available per patient was 11 (range, 5–20, depending on whether the specimen was from a lobe or an entire lung). The mean diameter of the tumor on the fixed specimens was 3.1 cm. The tumor was not considered in the quantification of emphysema. In 52 cases, the specimen was obtained at lobectomy or bilobectomy. In the remaining six cases, a pneumonectomy was performed; in these cases, the tumor-free lobe was used for quantification. Quantification was performed with entire lung in the four patients who underwent transplantation.

When considered on a lobar basis, the analyzed material can be described as follows: entire lung in 10 (16%) patients, right superior lobe in 23 (37%), middle lobe in two (3%), right inferior lobe in six (10%), left superior lobe in 13 (21%), left inferior lobe in four (6%), and middle lobe and right inferior lobe obtained at bilobectomy in four (6%). Thus, material from 10 (16%) entire lungs, 36 (58%) superior lobes, two (3%) middle lobes, and 14 (23%) inferior and/or inferior and middle lobes was available for analysis.

The area macroscopically occupied by emphysema was measured on the paper-mounted lung sections by using a computer-assisted method that had previously been validated (6). Lung sections were divided into 7 x 7-cm fields, digitized, and stored by using a personal computer. Image thresholds were selected by the operator so that the highest gray levels corresponded to emphysematous spaces and the lower gray levels corresponded to lung structures. Before the calculations were performed, the vessels and bronchi were digitally removed from images. The number of pixels in the area with emphysema and the number of pixels in the total lung area on the digitized image of the paper-mounted lung section were counted separately with the aid of the computer. The results obtained per field were added, and the overall relative area occupied by regions with subthreshold attenuation values was calculated and expressed as a percentage for the entire surgical specimen and, thus, corresponded to the extent of macroscopic emphysema in this specimen.

CT densitometry.—Densitometric measurements were performed with PULMO CT software (Siemens Medical Systems). This software automatically recognizes the lungs, traces lung contours, determines histograms of attenuation values, and calculates the lung area occupied by pixels with a predetermined range of attenuation values (13). After performing calculations separately with each scanning level, the program also provides results based on the total number of scans. In this study, the percentage of relative lung area occupied by attenuation values of less than -950 HU (RA950) was calculated. This threshold has been macroscopically and microscopically validated for thin-section CT studies of emphysema (7,8). On the basis of this threshold, RA950 was calculated for the lobe or the lung to be resected (lobe RA950) and for both lungs (lung RA950).

Subjective Assessment
All images were subjectively assessed by three readers (A.A.B., C.K., P.A.G.). Reader 1 was a board-certified chest radiologist skilled in CT of the thorax but with no supplementary experience in the assessment of emphysema. Reader 2 was a board-certified chest radiologist who specialized in the imaging of emphysema. During an extensive research project on the CT assessment of emphysema, he had examined hundreds of CT scans and Gough-Wenthworth sections of emphysematous lungs. He also was experienced in correlating the CT morphology of emphysema to densitometric and histologic data. Reader 3 was a student near completion of medical school, with neither special training in radiology nor previous radiologic experience.

All images were evaluated twice (reading sessions A and B) by each of the three readers. During the second reading (session B), images were evaluated in reverse order. Reader 2 and reader 3 reread the images 3 weeks after the first reading, and reader 1 reread the images 5 days after the first reading. All three readers were blinded to any clinical, functional, or morphologic information.

The system used to score the extent of emphysema on the CT scans was adapted from prior work by Goddard et al (14) and Bergin et al (1). All sections above the level of the diaphragm were assessed. Each CT section was assessed individually, and the right and left lungs were graded separately according to the percentage area that demonstrated changes suggestive of emphysema. These changes were areas of low attenuation, areas of lung destruction, and areas of vascular disruption (Fig 1). A score of 0 was assigned if there was no abnormality. A score of 1 was assigned if less than 25% of the parenchyma in the section was considered to show abnormalities suggestive of emphysema; a score of 2 was assigned if 25%–50% of the parenchyma had abnormalities suggestive of emphysema; a score of 3 was assigned if 51%–75% of the parenchyma had abnormalities suggestive of emphysema; and a score of 4 was assigned if more than 75% of the parenchyma had abnormalities suggestive of emphysema. This yielded maximum possible scores of 4 for the left side and 4 for the right side, for a maximum possible total score of 8 for each section. To obtain the maximum possible score, the maximum score per section was then multiplied by the number of sections. The final score was calculated as a percentage of the maximum possible score.



View larger version (125K):
[in this window]
[in a new window]
 
Figure 1a. Thin-section CT scans in a patient with emphysema, obtained (a) at the level of the carina and (b) at the level of the lower lobes. Areas of low attenuation (curved arrows), zones of vascular disruption (solid straight arrow), and areas of lung destruction (open arrows) are seen. All scans in this patient were assigned a score of 3 or 4 (>50% of lung parenchyma affected by emphysema) by all three readers. At objective quantification (lung RA950), only 18.9% of the parenchymal surface corresponded to emphysema.

 


View larger version (127K):
[in this window]
[in a new window]
 
Figure 1b. Thin-section CT scans in a patient with emphysema, obtained (a) at the level of the carina and (b) at the level of the lower lobes. Areas of low attenuation (curved arrows), zones of vascular disruption (solid straight arrow), and areas of lung destruction (open arrows) are seen. All scans in this patient were assigned a score of 3 or 4 (>50% of lung parenchyma affected by emphysema) by all three readers. At objective quantification (lung RA950), only 18.9% of the parenchymal surface corresponded to emphysema.

 
Prior to the first image reading, a consensus session was held. The aim of this session was to familiarize the readers with the scoring system as validated with objective measurements. Therefore, a radiologist not involved in later image readings selected 50 CT sections that demonstrated a percentage of macroscopic and densitometric emphysematous destruction that corresponded to each of the subjective scores (10 sections for each score). All sections were selected on the basis of both macroscopic morphometric and densitometric data. Scans used for the consensus session would not be included in later image readings. At a first session, all images were read without knowledge of macroscopic and densitometric results. Then, the results of macroscopic morphometry and densitometry were made available to the readers, and the CT images were reread.

Statistical Analyses
As a first step, we investigated the relationships between (a) the densitometrically obtained RA950 of the resected lobe or lung (lobe RA950) and the macroscopic morphometric measurements of emphysema, (b) the densitometrically obtained RA950 of both lungs (lung RA950) and macroscopic morphometric measurements of emphysema, and (c) lobe RA950 and lung RA950.

As a second step, for each of the three readers, we investigated the relationships between (a) the average subjective emphysema score determined at reading session A and the average subjective emphysema score determined at session B, (b) the average of the subjective emphysema scores from both reading sessions, and (c) the macroscopic morphometric measurements of emphysema and the densitometric results. Calculations of the average subjective emphysema scores from the two reading sessions were based on the total of scores obtained per individual CT section in the 62 patients, for both the right and the left lung. This resulted in a minimum of two by 12 evaluations and a maximum of two by 26 evaluations, depending on the apical-diaphragmatic distance in the individual patients.

Interobserver agreement was evaluated for the two reading sessions; intraobserver agreement also was investigated.

All data analyses were performed with the STATXACT 3 statistical software (Cytel, Cambridge, Mass). Relationships were investigated by using the Spearman correlation coefficient. Linear regression lines also were investigated. Differences between means and medians of lung RA950 and lobe RA950 were tested for statistical significance by using the paired Student t test for the means and the Wilcoxon test for paired data for the medians. The 95% CIs of the regression line intercepts for the mean and median lung RA950 and lobe RA950 values were computed. Interobserver agreement and intraobserver agreement were assessed with the weighted {kappa} statistic (15). The 95% CIs for the {kappa} statistics were calculated. The {kappa} values were interpreted on the basis of reports in the literature (15,16): A {kappa} value of less than 0.20 indicated poor agreement; a {kappa} value of 0.21–0.40, fair agreement; a {kappa} value of 0.41–0.60, moderate agreement; a {kappa} value of 0.61–0.80, good agreement; and a {kappa} value of 0.81–1.00, excellent agreement. Statistical significance for all tests was set at the P less than .05 level.


    RESULTS
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Correlation coefficients for the relationships among macroscopic morphometric measurements, lobe RA950, and lung RA950 are shown in Table 1. The 95% CIs of the intercepts for the linear regression lines for lobe RA950 and lung RA950 versus morphometric measurements were -0.13, 4.64 and -1.083, 4.933, respectively. Both confidence intervals included 0, which is suggestive of the absence of systematic overestimation of the extent of emphysema when assessed with densitometry, as compared with that assessed with macroscopic morphometry (Fig 2). The mean percentage area of lung macroscopically occupied by emphysema was 9.19% (range, 0.13%–50.81%). There was no difference between mean lung RA950 and mean lobe RA950 (P = .587, paired Student t test). There also was no difference between median lung RA950 and median lobe RA950 (P = .153, Wilcoxon test).


View this table:
[in this window]
[in a new window]
 
TABLE 1. Correlations between Lobe RA950, Lung RA950, and Macroscopic Morphometric Measurements in Patients with Emphysema
 


View larger version (16K):
[in this window]
[in a new window]
 
Figure 2a. Graphs show the correlation between macroscopic morphometric measurements and (a) lobe RA950 and (b) lung RA950. Both lobe RA950 and lung RA950 show good concordance with macroscopic morphometric results. The line in both graphs is the linear regression line.

 


View larger version (16K):
[in this window]
[in a new window]
 
Figure 2b. Graphs show the correlation between macroscopic morphometric measurements and (a) lobe RA950 and (b) lung RA950. Both lobe RA950 and lung RA950 show good concordance with macroscopic morphometric results. The line in both graphs is the linear regression line.

 
Table 2 shows the correlations between the subjective emphysema scores determined at reading sessions A and B and the correlations between the means of those subjective scores and the macroscopic morphometric and densitometric measurements. Although strong statistically significant correlations were seen between the subjective scores determined at the two reading sessions for the three readers, correlations between macroscopic measurements and the average subjective scores were weak. The best correlation between subjective scores and macroscopic morphometric measurements was achieved by the least experienced reader.


View this table:
[in this window]
[in a new window]
 
TABLE 2. Correlations between Morphometric, Densitometric, and Subjective Scores from Two Reading Sessions
 
The 95% CIs of the intercepts of the linear regression lines for subjective scores and densitometric results and for subjective scores and macroscopic morphometric measurements are listed in Table 3. All but one of the CIs did not contain 0, which is suggestive of systematic subjective overestimation of emphysema when compared with objective measurements (Figs 3, 4).


View this table:
[in this window]
[in a new window]
 
TABLE 3. Linear Regression Results: Subjective Scores and Densitometric and Morphometric Measurements
 


View larger version (18K):
[in this window]
[in a new window]
 
Figure 3a. Graphs show the correlation between macroscopic morphometric measurements and the average subjective emphysema scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 


View larger version (18K):
[in this window]
[in a new window]
 
Figure 3b. Graphs show the correlation between macroscopic morphometric measurements and the average subjective emphysema scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 


View larger version (17K):
[in this window]
[in a new window]
 
Figure 3c. Graphs show the correlation between macroscopic morphometric measurements and the average subjective emphysema scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 


View larger version (16K):
[in this window]
[in a new window]
 
Figure 4a. Graphs show the correlation between lung RA950 and the average subjective scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 


View larger version (16K):
[in this window]
[in a new window]
 
Figure 4b. Graphs show the correlation between lung RA950 and the average subjective scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 


View larger version (16K):
[in this window]
[in a new window]
 
Figure 4c. Graphs show the correlation between lung RA950 and the average subjective scores from (a) reader 1, (b) reader 2, and (c) reader 3. There is markedly increased scattering of the subjective scores around the regression lines as compared with those in Figure 2.

 
Results for interobserver agreement between the three readers for both reading sessions are summarized in Table 4. No substantial differences in interobserver agreement were seen between the first reading session and the second. All {kappa} values indicated moderate agreement. At both reading sessions, interobserver agreement was best between the most experienced and the least experienced reader.


View this table:
[in this window]
[in a new window]
 
TABLE 4. Interobserver Agreement about Scores Determined with CT Sections
 
Table 5 shows intraobserver agreement data for each reader between reading sessions A and B. Agreement was good for the least experienced reader and excellent for the two more experienced readers.


View this table:
[in this window]
[in a new window]
 
TABLE 5. Intraobserver Agreement between Reading Sessions for Subjective Emphysema Scoring Based on CT Sections
 

    DISCUSSION
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Given the medical and socioeconomic importance of pulmonary emphysema, the need for generally available and accurate quantitative grading systems for epidemiologic and clinical studies of this disease has been emphasized (1720). Although objective quantitative analysis of lung parenchyma may help eliminate the variability associated with subjective interpretation (21,22), direct visual observation and subjective visual grading were assumed to be nearly as precise as the technically more complicated methods of detecting and quantifying emphysema on CT scans (23). On the basis of the data from our study, we cannot confirm this statement. Our results revealed that, independent of the level of expertise of the individual reader, subjective visual grading of emphysema was inferior as compared with both the macroscopic reference standard and the objective method.

For all three readers, the correlation between subjective scores and macroscopic morphometric results was weaker than the correlation between objective densitometric and macroscopic morphometric measurements. It is notable that the least experienced reader achieved the best correlation between subjective grading and macroscopic morphometric measurement.

Because subjective assessment in our study was tested against an objective reference method, our correlations are difficult to compare with correlations between subjective visual assessment of CT images and subjective visual assessment of pathologic specimens, as described in prior investigations (24,24). Moreover, this comparison is hampered by major differences in the analysis of the radiologic material. Hruban et al (2) reported a correlation coefficient of 0.93 between visual assessment results and pathologic grade in 20 lungs.

Although the ideal number of CT sections needed for the quantification of emphysema remains unknown (7), only five scanning levels were analyzed per patient in the study of Hruban et al (2), whereas our readers analyzed all scans obtained from a complete CT examination. Kuwano et al (4), who reported a correlation coefficient of 0.68 for visual and pathologic grading in 42 patients, also assessed only five scanning levels per patient. In a study by Miller et al (3), who reported a correlation coefficient of 0.85 between visual assessment results and pathologic grade in 38 patients, the number of sections analyzed was even fewer: two sections per patient. Bergin et al (1) reported a correlation coefficient of 0.63 between visual scores and pathologic grade in 32 patients; Hruban et al, however, used 10-mm-collimation sections, whereas we used 1-mm-collimation thin sections. Müller et al (24) also used 10-mm-collimation sections and reported a correlation coefficient of 0.90 between visual and pathologic scores.

Because the above-mentioned studies were performed in an era when the acquisition time was slower than is currently the case, as long as 7 seconds (2) were needed for image acquisition. In contrast, the images in our study were obtained with the current, standard image-acquisition time of 1 second. Therefore, we speculate that some of the scans included in the above-mentioned studies would not fulfill current standards for image quality, owing to cardiac and respiratory motion. Overall, we suggest that the use of complete data sets, thin sections, and short acquisition times strengthens our results.

A notable aspect of our findings is the systematic subjective overestimation of emphysema. Indeed, the 95% CI did not include 0 for five of six comparisons between subjective and objective assessment results. Subjective overestimation of emphysema has been reported previously (4) but was attributed to the radiologic image-reading results of one of three readers, a chest physician who consistently overestimated emphysema. The other two readers, both radiologists, reportedly did not overestimate emphysema.

In contrast, our results demonstrated that there may be a risk for radiologists to subjectively overestimate emphysema, and that this overestimation may occur independent of the level of experience of the individual radiologist. It is notable that the authors of another investigation (3) reported underestimation of emphysema at subjective assessment. That study was, however, partially focused on mild emphysema, whereas our study included patients with emphysema of all grades of severity.

It is possible that subjective overestimation, as described in our study, could result from the fact that, under test conditions, readers have the tendency to err on the side of caution (25,26); that is, they may overestimate rather than underestimate emphysema. In this setting, overestimation could have become obvious because subjective grading was tested against an objective reference method, whereas in prior investigations (24), subjective visual grading was compared with a second subjective, semiquantitative ranking method. This could have masked overestimation, because a systematic reading error might have affected both subjective, semiquantitative readings to an equal amount.

Finally, if the use of CT findings resulted in overestimation of emphysema, this should have become evident in the correlation between densitometric and macroscopic morphometric results. No such overestimation was documented with the 95% CIs. In summary, this aspect of our results suggests that subjective over- or underestimation of emphysema, as previously reported (3), may be explained instead in terms of the overall subjective, semiquantitative approach used in previous investigations and should not be considered to be a systematic factor inherent to the CT analysis of emphysema.

Interobserver agreement in our study was surprisingly low. It is notable that the best agreement was obtained between the least experienced and the most experienced reader, whereas a slightly lower level of agreement was seen between the two more experienced readers. Although observer agreement was assessed with the weighted {kappa} test in our study, previous investigators applied various forms of correlation to evaluate this parameter. Thus, agreement that may have occurred due to chance was not taken into statistical account. Whereas Hruban et al (2) did not calculate any interobserver agreement, Bergin et al (1) reported interobserver agreement, as determined with correlation coefficients, that ranged from 0.76 to 0.85. Miller et al (3) used the interclass correlation coefficient and found interobserver agreement to be 0.80. Kuwano et al (4) reported interobserver agreement that ranged from 0.54 to 0.83, as assessed with the Spearman correlation coefficient.

Because direct comparison of these correlations with the {kappa} values obtained in our study is difficult, we can hardly comment on these data. Our results nevertheless strongly suggest that, contrary to the results of the previous studies, and independent of the expertise of the individual reader, lower interobserver agreement may compromise subjective grading of emphysema.

In contrast to the interobserver agreement results, our study yielded satisfactory results for intraobserver agreement, with the lowest intraobserver agreement reached by the least experienced reader. The excellent intraobserver agreement achieved by reader 1 may, in part, be explained by the fact that the time elapsed between the two reading sessions was shorter than that for the other two readers (27). However, except for this interval, no substantial differences were observed in intraobserver agreement for the two more experienced readers. The time between two reading sessions has, indeed, been shown (27,28) to be an influential factor with regard to the results obtained in these sessions. Previous studies (14), however, in which intraobserver agreement of 0.54–0.96 was reported, as evaluated with various forms of correlation, were not precise about the time frame of the readings. Although our results indicate that intraobserver agreement does not substantially hamper subjective grading of emphysema, it remains unclear whether this outweighs the overall poor interobserver agreement discussed earlier.

Potential drawbacks of our study should be mentioned. The initial step of our analysis included comparison of the two objective methods used in this study, namely, densitometry and macroscopic morphometry. In previous investigations (7), large overlaps in the distributions of densitometric and macroscopic measurements of emphysematous lungs were reported, which thus indicated the similarity of results obtained with these two methods. According to Bland and Altman (29), this strengthens the correlation coefficient by applying a complementary statistical parameter determined by the equality of means and the equality of medians. Also, the higher correlation of macroscopic measurements with lobe RA950 than with lung RA950 was not surprising, because the relative area was validated with results from the resected lobe and not with data from both lungs.

Furthermore, the comparison between macroscopic morphometric measurements (performed with the resected specimen) and subjective scores (performed with CT studies of the entire lungs) might appear to be problematic; however, it has been shown (30) that there is good concordance between the grade of emphysema for either the upper or lower lobe and that for the entire lung. This concordance was confirmed with our results, which showed no statistically significant differences between the mean and the median objective values as obtained with the resected lobes and with entire lung. An unambiguous solution to the above-mentioned issue could be achieved by performing CT-pathologic correlations in large autopsy series, but, mainly for practical reasons, such series do not exist.

Minor drawbacks could also have resulted from the fact that the agreement between the subjective assessment by the three observers was based on the individual scores obtained with all left and right CT sections in each patient. This analysis, thus, did not take into account the fact that the individual CT sections were obtained in the same patients. In addition, correlations between the two reading sessions were based on the calculated averages of the subjectively determined scores. Therefore, bias could have been introduced because similar averages could result from a series of different individual scores. Finally, most of the patients included in this study had been surgically treated for lung cancer, and only a few patients had macroscopically severe emphysema. The presence of severe emphysema probably would have been an obstacle to surgery in many of the patients. Therefore, it must remain open whether the correspondence between CT measurements and morphometric data might have been strengthened if a substantially larger number of patients with severe emphysema had been included in our sample.

Our results can be summarized as follows: Subjective emphysema scores, as obtained in this study, are not strongly correlated with results from either the objective macroscopic reference standard or the objective assessment with CT densitometry. Subjective scoring also suffers because of systematic overestimation of emphysema. Moreover, subjective grading is compromised owing to moderate interobserver agreement. Because these drawbacks are evident regardless of the expertise of the individual reader, our results suggest that subjective visual grading should be supplemented with more reliable objective methods whenever a precise, reader-independent quantification of emphysema is needed.


    Footnotes
 
Abbreviation: RA950 = relative lung area with attenuation values of less than -950 HU

Author contributions: Guarantors of integrity of entire study, A.A.B., P.A.G.; study concepts, P.A.G.; study design, A.A.B., P.A.G.; definition of intellectual content, A.A.B., V.D.M., P.A.G.; literature research, A.A.B., P.A.G.; clinical studies, A.A.B., C.K., P.A.G.; data acquisition, A.A.B., C.K., P.A.G.; data analysis, A.A.B., V.D.M., P.A.G.; statistical analysis, V.D.M.; manuscript preparation and review, A.A.B., V.D.M., P.A.G.; manuscript editing, A.A.B., P.A.G.


    References
 TOP
 Abstract
 Introduction
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 

  1. Bergin C, Müller NL, Nichols DM, et al. The diagnosis of emphysema: a computed tomographic–pathologic correlation. Am Rev Respir Dis 1986; 133:541-546.[Medline]
  2. Hruban RH, Mezian MA, Zerhouni EA, et al. High-resolution computed tomography of inflation-fixed lungs: pathologic-radiologic correlation of centrilobular emphysema. Am Rev Respir Dis 1987; 136:935-940.[Medline]
  3. Miller RR, Müller NL, Vedal S, Morrison NJ, Staples CA. Limitations of computed tomography in the assessment of emphysema. Am Rev Respir Dis 1989; 139:980-983.[Medline]
  4. Kuwano K, Matsuba K, Ikeda T, et al. The diagnosis of mild emphysema: correlation of computed tomography and pathology scores. Am Rev Respir Dis 1990; 141:169-178.[Medline]
  5. Thurlbeck WM, Dunnil MS, Hartung W, Heard BE, Heppleston AG, Ryder RC. A comparison of three methods of measuring emphysema. Hum Pathol 1972; 1:215-226.
  6. Gevenois PA, Zanen J, de Maertelaer V, De Vuyst P, Dumortier P, Yernault JC. Macroscopic assessment of pulmonary emphysema by image analysis. J Clin Pathol 1995; 48:318-322.[Abstract/Free Full Text]
  7. Gevenois PA, de Maertelaer V, De Vuyst P, Zanen J, Yernault JC. Comparison of computed density and macroscopic morphometry in pulmonary emphysema. Am J Respir Crit Care Med 1995; 152:653-657.[Abstract]
  8. Gevenois PA, De Vuyst P, de Maertelaer V, et al. Comparison of computed density and microscopic morphometry in pulmonary emphysema. Am J Respir Crit Care Med 1996; 154:187-192.[Abstract]
  9. Wallaert B, Gressier B, Marquette CH, et al. Inactivation of alpha 1–proteinase inhibitor by alveolar inflammatory cells from smoking patients with and without emphysema. Am Rev Respir Dis 1993; 147:1537-1543.[Medline]
  10. Gelb AF, Schein M, Kuei J, et al. Limited contribution of emphysema in advanced chronic obstructive pulmonary disease. Am Rev Respir Dis 1993; 147:1157-1161.[Medline]
  11. Gough J, Wenthworth JE. The use of thin sections of entire organs in morbid anatomical structures. J R Microsc Soc 1949; 69:231-235.
  12. Gevenois PA, Koob MC, Jacobovitz D, De Vuyst P, Yernault JC, Struyven J. Whole-lung sections for CT-pathologic correlations: modified Gough-Wenthworth technique. Invest Radiol 1993; 28:242-246.[Medline]
  13. Kalender WA, Fichte H, Bautz W, Skalej M. Semiautomatic evaluation procedures for quantitative CT of the lung. J Comput Assist Tomogr 1991; 15:248-255.[Medline]
  14. Goddard PR, Nicholson EM, Laszlo G, Watt I. Computed tomography in pulmonary emphysema. Clin Radiol 1982; 33:379-387.[Medline]
  15. Liebetrau AM. Measures of association Sage University Papers Series on Quantitative Applications in the Social Sciences, series 07-032, Newbury Park, Calif: Sage, 1983; 32-36.
  16. Altman DG. Practical statistics for medical research London, England: Chapman & Hall, 1994; 403-409.
  17. Snider GL. Emphysema: the first two centuries—and beyond: a historical overview, with suggestions for future research—part 1. Am Rev Respir Dis 1992; 146(6 pt 1):1334-1343.[Medline]
  18. Snider GL. Emphysema: the first two centuries—and beyond: a historical overview, with suggestions for future research—part 2. Am Rev Respir Dis 1992; 146:1615-1622.[Medline]
  19. Morgan MD. Detection and quantification of pulmonary emphysema by computed tomography: a window of opportunity (editorial). Thorax 1992; 47:1001-1004.[Free Full Text]
  20. Flenley DC. Diagnosis and follow-up of emphysema. Eur Respir J 1990; 3(suppl 9):5S-8S.
  21. Thurlbeck WM, Müller NL. Emphysema: definition, imaging, and quantification. AJR 1994; 163:1017-1025.[Abstract/Free Full Text]
  22. Gevenois PA, Yernault JC. Can computed tomography quantify pulmonary emphysema?. Eur Respir J 1995; 5:843-848.
  23. Stern EJ, Frank MS. CT of the lung in patients with pulmonary emphysema: diagnosis, quantification, and correlation with pathologic and physiologic findings. AJR 1994; 162:791-798.[Abstract/Free Full Text]
  24. Müller NL, Miller RR, Abboud RT. "Density mask": an objective method to quantitate emphysema using computed tomography. Chest 1988; 94:782-787.[Abstract/Free Full Text]
  25. Brooks KW, Trueblood JH, Kearfott KJ. Subjective evaluations of mammographic accreditation phantom images by three observer groups. Invest Radiol 1994; 29:42-47.[Medline]
  26. Rackow PL, Spitzer VM, Hendee WR. Detection of low-contrast signals: a comparison of observers with and without radiology training. Invest Radiol 1987; 22:311-314.[Medline]
  27. Gur D, Rockette HE, Good WF, et al. Effect of observer instruction on ROC study of chest images. Invest Radiol 1990; 25:230-234.[Medline]
  28. Sabba C, Merkel C, Zoli M, et al. Interobserver and interequipment variability of echo-Doppler examination of the portal vein: effect of a comparative training programme. Hepatology 1995; 21:428-433.[Medline]
  29. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307-310.[Medline]
  30. Wright JL, Wiggs B, Paré PD, Hogg JC. Ranking the severity of emphysema on whole lung slices: concordance of upper lobe, lower lobe, and entire lung ranks. Am Rev Respir Dis 1986; 133:930-931.[Medline]



This article has been cited by other articles:


Home page
Phil Trans R Soc AHome page
M. Plotkowiak, K. Burrowes, J. Wolber, C. Buckley, R. Davies, F. Gleeson, D. Gavaghan, and V. Grau
Relationship between structural changes and hyperpolarized gas magnetic resonance imaging in chronic obstructive pulmonary disease using computational simulations with realistic alveolar geometry
Phil Trans R Soc A, June 13, 2009; 367(1896): 2347 - 2369.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
S. J. Copley, A. U. Wells, K. E. Hawtin, D. J. Gibson, J. M. Hodson, A. E. T. Jacques, and D. M. Hansell
Lung Morphology in the Elderly: Comparative CT Study of Subjects over 75 Years Old versus Those under 55 Years Old
Radiology, May 1, 2009; 251(2): 566 - 573.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
K. L. Irion, E. Marchiori, B. Hochhegger, N. da Silva Porto, J. da Silva Moreira, C. E. Anselmi, J. A. Holemans, and P. O. Irion
CT Quantification of Emphysema in Young Subjects with No Recognizable Chest Disease
Am. J. Roentgenol., March 1, 2009; 192(3): W90 - W96.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
A. Busacker, J. D. Newell Jr, T. Keefe, E. A. Hoffman, J. C. Granroth, M. Castro, S. Fain, and S. Wenzel
A Multivariate Analysis of Risk Factors for the Air-Trapping Asthmatic Phenotype as Measured by Quantitative CT Analysis
Chest, January 1, 2009; 135(1): 48 - 56.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
J. W. Song, W.-J. Koh, K. S. Lee, J. Y. Lee, M. J. Chung, T. S. Kim, and O J. Kwon
High-Resolution CT Findings of Mycobacterium avium-intracellulare Complex Pulmonary Disease: Correlation with Pulmonary Function Test Results
Am. J. Roentgenol., October 1, 2008; 191(4): W160 - W166.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
C. Keyzer, S. Pargov, D. Tack, V. Creteur, P. Bohy, V. De Maertelaer, and P. A. Gevenois
Normal Appendix in Adults: Reproducibility of Detection with Unenhanced and Contrast-Enhanced MDCT
Am. J. Roentgenol., August 1, 2008; 191(2): 507 - 514.
[Abstract] [Full Text] [PDF]


Home page
Proc Am Thorac SocHome page
G. R. Washko, E. Hoffman, and J. J. Reilly
Radiographic Evaluation of the Potential Lung Volume Reduction Surgery Candidate
Proceedings of the ATS, May 1, 2008; 5(4): 421 - 426.
[Abstract] [Full Text] [PDF]


Home page
JNMHome page
J. J. Mathews, A. H. Maurer, R. M. Steiner, N. Marchetti, G. Criner, J. P. Gaughan, and H. O. Coxson
New 133Xe Gas Trapping Index for Quantifying Severe Emphysema Before Partial Lung Volume Reduction
J. Nucl. Med., May 1, 2008; 49(5): 771 - 775.
[Abstract] [Full Text] [PDF]


Home page
Eur Respir JHome page
M. Cazzola, W. MacNee, F. J. Martinez, K. F. Rabe, L. G. Franciosi, P. J. Barnes, V. Brusasco, P. S. Burge, P. M. A. Calverley, B. R. Celli, et al.
Outcomes for COPD pharmacological trials: from lung function to biomarkers
Eur. Respir. J., February 1, 2008; 31(2): 416 - 469.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
R. Yuan, J. R. Mayo, J. C. Hogg, P. D. Pare, A. M. McWilliams, S. Lam, and H. O. Coxson
The Effects of Radiation Dose and CT Manufacturer on Measurements of Lung Densitometry
Chest, August 1, 2007; 132(2): 617 - 623.
[Abstract] [Full Text] [PDF]


Home page
ThoraxHome page
T Higuchi, A Reed, T Oto, L Holsworth, S Ellis, M J Bailey, T J Williams, and G I Snell
Relation of interlobar collaterals to radiological heterogeneity in severe emphysema
Thorax, May 1, 2006; 61(5): 409 - 413.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
G. Camiciottoli, M. Bartolucci, N. M. Maluccio, C. Moroni, M. Mascalchi, C. Giuntini, and M. Pistolesi
Spirometrically Gated High-Resolution CT Findings in COPD: Lung Attenuation vs Lung Function and Dyspnea Severity
Chest, March 1, 2006; 129(3): 558 - 564.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
A. Madani, J. Zanen, V. de Maertelaer, and P. A. Gevenois
Pulmonary Emphysema: Objective Quantification at Multi-Detector Row CT--Comparison with Macroscopic and Microscopic Morphometry
Radiology, March 1, 2006; 238(3): 1036 - 1043.
[Abstract] [Full Text] [PDF]


Home page
Ann. Thorac. Surg.Home page
K. Ueda, Y. Kaneda, M. Sudo, J. Mitsutaka, T.-S. Li, K. Suga, N. Tanaka, and K. Hamano
Quantitative Computed Tomography Versus Spirometry in Predicting Air Leak Duration After Major Lung Resection for Cancer
Ann. Thorac. Surg., November 1, 2005; 80(5): 1853 - 1858.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
J. Zaporozhan, S. Ley, R. Eberhardt, O. Weinheimer, S. Iliyushenko, F. Herth, and H.-U. Kauczor
Paired Inspiratory/Expiratory Volumetric Thin-Slice CT Scan for Emphysema Analysis: Comparison of Different Quantitative Evaluations and Pulmonary Function Test
Chest, November 1, 2005; 128(5): 3212 - 3220.
[Abstract] [Full Text] [PDF]


Home page
ICVTSHome page
K. Ueda, Y. Kaneda, M. Sudou, M. Jinbo, T.-S. Li, K. Suga, N. Tanaka, and K. Hamano
Prediction of hypoxemia after lung resection surgery
Interactive CardioVascular and Thoracic Surgery, April 1, 2005; 4(2): 85 - 89.
[Abstract] [Full Text] [PDF]


Home page
RadioGraphicsHome page
J.-M. Kuhnigk, V. Dicken, S. Zidowitz, L. Bornemann, B. Kuemmerlen, S. Krass, H.-O. Peitgen, S. Yuval, H.-H. Jend, W. S. Rau, et al.
Informatics in Radiology (infoRAD): New Tools for Computer Assistance in Thoracic CT. Part 1. Functional Analysis of Lungs, Lung Lobes, and Bronchopulmonary Segments
RadioGraphics, March 1, 2005; 25(2): 525 - 536.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
I. Orlandi, C. Moroni, G. Camiciottoli, M. Bartolucci, M. Pistolesi, N. Villari, and M. Mascalchi
Chronic Obstructive Pulmonary Disease: Thin-Section CT Measurement of Airway Wall Thickness and Lung Attenuation
Radiology, February 1, 2005; 234(2): 604 - 610.
[Abstract] [Full Text] [PDF]


Home page
ThoraxHome page
S B Shaker, T Stavngaard, J Stolk, B Stoel, and A Dirksen
{alpha}1-Antitrypsin deficiency {middle dot} 7: Computed tomographic imaging in {alpha}1-antitrypsin deficiency
Thorax, November 1, 2004; 59(11): 986 - 991.
[Abstract] [Full Text] [PDF]


Home page
Eur Respir JHome page
J.D. Newell Jr, J.C. Hogg, and G.L. Snider
Report of a workshop: quantitative computed tomography scanning in longitudinal studies of emphysema
Eur. Respir. J., May 1, 2004; 23(5): 769 - 775.
[Abstract] [Full Text] [PDF]


Home page
ThoraxHome page
H O Coxson, K P Whittall, Y Nakano, R M Rogers, F C Sciurba, R J Keenan, and J C Hogg
Selection of patients for lung volume reduction surgery using a power law analysis of the computed tomographic scan
Thorax, June 1, 2003; 58(6): 510 - 514.
[Abstract] [Full Text] [PDF]


Home page
ThoraxHome page
N L Muller and H Coxson
Chronic obstructive pulmonary disease * 4: Imaging the lungs in patients with chronic obstructive pulmonary disease
Thorax, November 1, 2002; 57(11): 982 - 985.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
K. Cederlund, U. Tylen, L. Jorfeldt, and P. Aspelin
Classification of Emphysema in Candidates for Lung Volume Reduction Surgery* : A New Objective and Surgically Oriented Model for Describing CT Severity and Heterogeneity
Chest, August 1, 2002; 122(2): 590 - 596.
[Abstract] [Full Text] [PDF]


Home page
NEJMHome page
National Emphysema Treatment Trial Research Group
Patients at High Risk of Death after Lung-Volume-Reduction Surgery
N. Engl. J. Med., October 11, 2001; 345(15): 1075 - 1083.
[Abstract] [Full Text] [PDF]


Home page
Eur Respir JHome page
A. Madani, C. Keyzer, and P.A. Gevenois
Quantitative computed tomography assessment of lung structure and function in pulmonary emphysema
Eur. Respir. J., October 1, 2001; 18(4): 720 - 730.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Respir. Crit. Care Med.Home page
S. BALDI, M. MINIATI, C. R. BELLINA, L. BATTOLLA, G. CATAPANO, E. BEGLIOMINI, D. GIUSTINI, and C. GIUNTINI
Relationship between Extent of Pulmonary Emphysema by High-resolution Computed Tomography and Lung Elastic Recoil in Patients with Chronic Obstructive Pulmonary Disease
Am. J. Respir. Crit. Care Med., August 15, 2001; 164(4): 585 - 589.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
D. S. Gierada, R. D. Yusen, T. K. Pilgram, L. Crouch, R. M. Slone, K. T. Bae, S. S. Lefrak, and J. D. Cooper
Repeatability of Quantitative CT Indexes of Emphysema in Patients Evaluated for Lung Volume Reduction Surgery
Radiology, August 1, 2001; 220(2): 448 - 454.
[Abstract] [Full Text] [PDF]


Home page
Eur Respir JHome page
H. De Raeve, J.A. Verschakelen, P.A. Gevenois, P. Mahieu, G. Moens, and B. Nemery
Observer variation in computed tomography of pleural lesions in subjects exposed to indoor asbestos
Eur. Respir. J., May 1, 2001; 17(5): 916 - 921.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
K. R. Flaherty, E. A. Kazerooni, J. L. Curtis, M. Iannettoni, L. Lange, M. A. Schork, and F. J. Martinez
Short-term and Long-term Outcomes After Bilateral Lung Volume Reduction Surgery : Prediction by Quantitative CT
Chest, May 1, 2001; 119(5): 1337 - 1346.
[Abstract] [Full Text] [PDF]


Home page
ChestHome page
R. M. Rogers, H. O. Coxson, F. C. Sciurba, R. J. Keenan, K. P. Whittall, and J. C. Hogg
Preoperative Severity of Emphysema Predictive of Improvement After Lung Volume Reduction Surgery : Use of CT Morphometry
Chest, November 1, 2000; 118(5): 1240 - 1247.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
M. Salerno, E. E. de Lange, T. A. Altes, J. D. Truwit, J. R. Brookeman, and J. P. Mugler III
Emphysema: Hyperpolarized Helium 3 Diffusion MR Imaging of the Lungs Compared with Spirometric Indexes—Initial Experience
Radiology, January 1, 2002; 222(1): 252 - 260.
[Abstract] [Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Bankier, A. A.
Right arrow Articles by Gevenois, P. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Bankier, A. A.
Right arrow Articles by Gevenois, P. A.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE