HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Gollub, M. J. Articles by Castellino, R. A. Search for Related Content PubMed PubMed Citation Articles by Gollub, M. J. Articles by Castellino, R. A.

Clinical Importance of Reinterpretation of Body CT Scans Obtained Elsewhere in Patients Referred for Care at a Tertiary Cancer Center

¹ Department of Radiology, Memorial Sloan-Kettering Cancer Center, 1275 York Ave, New York, NY 10021.

	Abstract

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 
PURPOSE: To determine the frequency and clinical importance of discordance between the initial interpretation of computed tomographic (CT) scans of the body and subsequent interpretations in patients with biopsy-proved cancer.

MATERIALS AND METHODS: The initial and reinterpretation reports for 213 CT scans of the body submitted for official review were compared independently by two radiologists. Sixty-nine sets of reports were excluded because the reviewing radiologists and the outside radiologists had prior CT scans from differing dates to use for comparison. One set of reports was excluded because of lack of clinical follow-up.

RESULTS: The interpretations were graded as "agree" in 90 patients (63%), "major disagreement" in 24 patients (17%), and "minor disagreement" in 29 patients (20%). A theoretic change in treatment could have occurred in nine of 53 cases of disagreement (17%). An actual change in treatment occurred in five of 53 cases of disagreement (9%).

CONCLUSION: Discordant interpretations were frequent (53 of 143 cases [37%]), were often major (24 of 143 cases [17%]), and resulted in actual treatment changes in five of all 143 cases (3%). Reinterpretation of body CT scans can have a substantial effect on the clinical care of individual patients with proved malignancy.

Index terms: Computed tomography (CT), clinical effectiveness • Diagnostic radiology, observer performance • Images, interpretation • Radiology and radiologists, outcomes studies

	Introduction

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 
Patients referred to tertiary care medical centers for evaluation frequently arrive with various imaging studies that were obtained elsewhere. At our comprehensive cancer center, these outside radiologic studies may be filed with other imaging studies to be used later for comparison; "curbside consultations" may be requested by our clinicians; and/or official interpretation (ie, reinterpretation) with issuance of a written report may be requested.

When official interpretations are requested, a copy of the initial report is required so that we may compare interpretations and ensure that no finding noted in the initial report is overlooked, among other reasons (1). We have occasionally noted marked differences in these interpretations and wondered about the importance of such differences and how often they might affect patient care. Therefore, we performed a retrospective study to determine how often discordant interpretations arose; whether such discordant interpretations were major or minor disagreements; and, most important, how often patient care might have changed or did change as a result of the reinterpretation.

We did not intend to compare the accuracy of one reading with that of another by using a standard of reference, such as histopathologic analysis, nor to compare the accuracies or interobserver variabilities of our radiologists and of other radiologists. Instead, we performed this study to investigate the frequency and magnitude of discordance and the effects on the treatment of patients with biopsy-proved cancer.

	MATERIALS AND METHODS

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 
In 1995, 1,877 computed tomographic (CT) scans of the body (chest, abdomen, and/or pelvis) that were obtained and interpreted at outside radiology facilities were submitted to our department for official review (ie, reinterpretation with issuance of an official report). Copies of the original radiologist's report were always available to our radiologists at the time of reinterpretation (1). From a computer-generated list of these cases (sorted numerically by procedure code), 213 sets of reports (initial report and report generated at our institution) were randomly obtained for review. Sixty-nine cases were excluded because the outside radiologists and our radiologists had access to prior studies that were not of the same date, a discrepancy that could affect interpretation (eg, different assessments of change in lesion size). In an additional patient, results of clinical follow-up were not available; lack of these results precluded assessment of the potential effect on clinical care.

The remaining 143 cases formed the basis of our study. All but one patient were adults with biopsy-proved cancer. One child with a neuroblastoma was included in the study. The mean age of the patients was 50.8 years ± 18.6 (SD) (range, 1–89 years). The most common malignancies were lymphoma (n = 52 [36%]), breast carcinoma (n = 17 [12%]), sarcoma (n = 10 [7%]), and prostate carcinoma (n = 9 [6%]).

The initial reports were independently compared with the reinterpretation reports by two radiologists and graded as "agree," "minor disagreement," or "major disagreement"; the cause of disagreement was specified as "mass," "lymphadenopathy," "liver mass," "lung nodules," or "other." A mass was defined as an abnormal soft-tissue structure. Lymphadenopathy was defined as at least one node 1 cm or larger in short-axis diameter or an increased number of normal-sized nodes. More than one cause of disagreement could be listed, but a report could be assigned to only one category (ie, either major disagreement, minor disagreement, or agree). If both major and minor disagreements occurred in a given case, the report was categorized as a major disagreement.

A minor disagreement was one that, in the opinion of both reviewers, was considered unlikely to alter patient care (eg, fatty liver, nonobstructive renal calculus, additional enlarged lymph node in the presence of widespread lymphadenopathy). A major disagreement was one that was considered capable of altering patient care (eg, hepatic metastases, disease progression, deep venous thrombosis). If there was lack of consensus between the two radiologists on whether a disagreement was present or in the classification as "major" versus "minor," then a third radiologist reviewed both reports to achieve consensus.

All disagreements were reviewed with the referring clinician to determine the theoretic and actual effects on patient care. Appointments were made with the clinicians, and their secretaries were asked to pull the charts for review so that the clinician could familiarize himself or herself with the case. In certain instances, the initial review by the clinician was performed by means of recall (eg, if the clinician thought he or she knew the patient's clinical information well). However, acquisition of further information or repeat questioning usually involved reference to the chart. A theoretic effect was considered present if a discordant interpretation noted on the reinterpretation report could have altered patient care but did not because (a) the patient declined treatment or had been evaluated on a consultation basis but treated elsewhere or (b) the attending clinician thought that the finding was equivocal or did not fit the clinical picture. Categories of effect were "initiate," "terminate," or "change" therapy, biopsy, or other or "obtain" or "cancel" additional diagnostic tests. "Initiate" was defined as beginning another therapy. "Change" was defined as altering the therapeutic approach (eg, altered dose, dose schedule, type of chemotherapeutic agent, radiation therapy ports, or surgical approach).

	RESULTS

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 
The 143 sets of reports were on various combinations of body CT scans (79 thoracic, 119 abdominal, 105 pelvic). Intravenously administered contrast material was used in 118 studies (83%). Reports were reviewed by only two radiologists in 126 cases (88%) and by a third radiologist in 17 cases (12%). In 53 cases (37%), there was a major (n = 24 [17%]) or minor (n = 29 [20%]) disagreement between the initial and reinterpretation reports (Table).

In four instances, abnormalities identified in the original report that were considered important were reinterpreted as being false-positive findings.

Minor Disagreement
The most frequent category of minor disagreement was also failure to report lymphadenopathy (n = 13), followed by failure to report an extrahepatic mass (n = 5), a hepatic mass (n = 3), or pulmonary nodules (n = 1). In two cases of minor disagreement, both of which involved lymphadenopathy, treatment actually changed. In a patient with prostate carcinoma, a few small pelvic nodes were identified in the initial report, but the importance of this finding was not indicated; however, the reinterpretation report included a qualitative assessment that the nodes, although subcentimeter in diameter, were compatible with metastases because of the increased number of nodes. On the basis of the latter report, the patient received hormonal therapy for 2 years. In the second case, the reinterpretation report disagreed with the initial report, which identified abdominal disease in a patient with cervical node lymphoma. This case was treated as stage IIa disease instead of stage IIIa disease (chemotherapy was discontinued, and mantle radiation therapy was administered).

In three instances, abnormalities identified in the original report that were considered minor were reinterpreted as being false-positive findings.

Changes in Treatment
Because the reinterpretation report is used by our clinicians in treating patients, a change in treatment would theoretically have occurred in nine of 53 cases of disagreement (17%) (four of 24 cases of major disagreement [17%], five of 29 cases of minor disagreement [17%]) and in nine of all 143 cases (6%) had the patient been treated at our institution. However, actual changes in treatment occurred in only five of 53 cases of disagreement (9%) (three of 24 cases of major disagreement [12%], two of 29 cases of minor disagreement [7%]) and in five of all 143 cases (3%) (Table).

Actual changes in treatment occurred in two patients with prostate cancer and one each with melanoma, ovarian cancer, and lymphoma. Treatment changes involved the administration of chemotherapy (n = 3), the initiation of surgery (n = 1), or the initiation of radiation therapy (n = 1). In nine patients (five with lymphoma, two with lung cancer, and one each with breast cancer and melanoma), theoretic changes did not result in actual changes because the patient refused treatment (n = 2) or was treated elsewhere (n = 3) or because the attending clinician believed the radiologic findings were equivocal or inconsistent with the clinical picture (n = 4). For example, in a patient with lung cancer, the reinterpretation report described hilar tumor involvement whereas the initial report did not. This finding would have changed the surgical approach from a lateral to a frontal thoracotomy. In another example, in a patient with breast cancer, a hepatic lesion was noted in the reinterpretation report but not in the initial report. Had the patient been treated at our institution, this finding would have been evaluated with biopsy, resected, or further investigated with magnetic resonance (MR) imaging of the liver; however, the patient was treated elsewhere.

	DISCUSSION

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 
Requests for second opinions are commonplace in the practice of medicine and often occur in the form of curbside consultations (2). However, official reinterpretation of radiologic studies is often requested because treatment in many institutions will be based on the reinterpretation report. We are aware of no published study that specifically addresses the rate of discordance of radiologic second opinions of body CT scans and the effects on treatment of patients with biopsy-proved cancer. We found a 17% rate of major disagreements and a 20% rate of minor disagreements. Treatment actually changed in 3% of all patients and theoretically changed in 6% of all patients.

The goal of this study was not to determine which report (initial report versus reinterpretation report) was correct, nor was the study a test of second readings. In fact, we had the advantage of seeing the initial report at the time of our official reinterpretation and thus were aware of all of the findings noted by the outside radiologist in the initial report. Instead, because the clinicians used the results of the reinterpretation reports for patient care, we sought to determine how frequently our reinterpretation report was discordant with the initial report and how often these findings changed patient care. Thus, no attempt was made to validate either report with clinical follow-up or histopathologic examination.

A recent study (3) investigated the usefulness of expert interpretation in oncologic imaging of patients who did not necessarily have a biopsy-proved diagnosis of cancer. In such patients, the actual diagnosis could change to a noncancer diagnosis by virtue of the reinterpretation. In contrast, all of the patients in our study had biopsy-proved malignancy. A change in diagnosis was therefore not a possibility; changes in stage or in severity of disease were the only discrepancies that could be found.

The results of our study are similar to those of a prospective review by radiologists of the results of 100 MR imaging examinations (4). In that study, a discordance rate of 39% was found; major disagreements accounted for 23% of discordant cases, minor disagreements for 16%. A similar result was found in a review performed at a university medical center of outside thoracic imaging studies and the associated reports for 111 cancer patients; major discrepancies were found in 25% of cases (5). In a study of the results of MR imaging examinations of the lower extremity, Piccoli et al (6) found discordant interpretations that could have affected treatment in up to 42% of cases. In fact, at a given institution, interpretational variability for abdominal CT scans may lead to discrepancies with subsequent change in treatment in up to 13% of patients (7).

The most common category of major disagreement in our study was lymphadenopathy (14 cases). In the majority of these cases, the initial report did not identify or misinterpreted lymphadenopathy. Bechtold et al (7) studied the interobserver variation in interpretation of 607 CT scans by radiologists from a single institution. There were 56 interpretive errors; lymphadenopathy accounted for two of the seven errors that could have affected treatment. Bollen et al (8) found fair interobserver variability among radiologists ( coefficient = 0.38) (9) for assessment of nodal status in the staging of lung cancer. Such variation exists, in part, because of variability in the interpreters' measurements and size criteria, as well as in subjective degrees of suspicion based on the locations, appearances, and multiplicity of nodes. We agree with Müller and Evans (10) that use of the criterion of 1-cm short-axis diameter for best reliability in measurement of lymph nodes is a concept that is difficult to grasp. In fact, it has recently been shown that lowering the threshold of suspicion for lymphadenopathy on CT scans to below 1 cm in diameter may be appropriate in testicular cancer (11). Also, use of a single size criterion for lymph nodes in various anatomic locations is probably simplistic; different mediastinal lymph node groups have been shown to have different normal sizes (12).

Several biases may have affected our results. First, the classification of discordance as clinically important is a subjective judgement (7). Radiologists are not always aware of what constitutes a major discrepancy with respect to the clinical implications because we may not have knowledge of disease outside the area being imaged (eg, known brain metastases when a body CT scan is being reviewed) that would affect the staging. Under certain circumstances, a radiologically important finding may not affect patient care. In our patients with known malignancies, many of whom have known metastases, failure to detect an additional enlarged lymph node, for example, may make little difference in treatment.

Second, selection bias may have occurred because submission of CT studies by our clinicians is not random. The clinicians who participated in the study explained that the most common reason for requesting an official reinterpretation was treatment of the patient at our institution according to protocol. Other reasons, in decreasing order of frequency, were because the patient's treatment would change; the patient was to be followed up at our institution; the clinician was unable to interpret the report; there was a specific question (eg, query lymph nodes in pelvis); results of the curbside consultation were discrepant with the outside reading; or miscellaneous (eg, the patient requested a second opinion, the clinician needed to make a decision, the clinician was not confident in the outside reading).

Third, review of a patient's clinical course with the clinician, although performed in part by means of recall, in most cases included chart review 18–30 months after patient care. This arrangement could have introduced bias because new information about the case would have become known in the intervening period (eg, progressive downhill course or marked improvement); such new information could have changed the perspective on the importance of certain findings in retrospect. Errors due to inaccurate recall could also have introduced bias.

In summary, among 143 cases of biopsy-proved malignancy referred to a cancer center in which an official interpretation of a body CT scan was requested, the reinterpretation was discordant with the outside radiologist's report in 53 cases (37%). Seventeen percent of the interpretations were deemed major disagreements that could have resulted in a change in patient care. Actual change in treatment occurred in only five of all 143 cases (3%) or in five of 53 cases of disagreement (9%). Although discordant interpretations occurred in more than one-third of the cases, the effect on treatment was small. The additional findings, which were accepted by our clinicians, did not often change treatment, probably because of the advanced stage of disease in many of the patients and the limited treatment options available to them. In some cases, the effect on clinical care was substantial. Therefore, we believe that the reinterpretation of body CT scans is beneficial, despite the resultant added work for our staff. Future studies would be helpful to determine criteria for selecting a subset of all body CT scans that would be most likely to yield important information at reinterpretation.

	Footnotes

 
Address reprint requests to M.J.G.

From the 1997 RSNA scientific assembly.

Author contributions: Guarantor of integrity of entire study, M.J.G.; study concepts and design, M.J.G., D.M.P., R.A.C.; definition of intellectual content, M.J.G., D.M.P., R.A.C., A.M.B.; literature research, M.J.G., R.A.C.; data acquisition and analysis, M.J.G., D.M.P., A.M.B., R.A.C.; statistical analysis, A.P., M.J.G.; manuscript preparation, M.J.G.; manuscript editing and review, M.J.G., D.M.P., A.M.B., R.A.C.

Received January 14, 1998; revision requested April 2, 1998; revision received June 19, 1998; accepted August 24, 1998.

	References

TOP Abstract Introduction MATERIALS AND METHODS RESULTS DISCUSSION References

 

1. Castellino RA. Official review of outside studies (opinion). Invest Radiol 1994; 1:80-81.
2. Farrin MA, Janssen DA. Curbside consultations: a closer look at a common practice (commentary). JAMA 1996; 275:145-147.[Medline]
3. Hricak H, Kalbhen CL, Scheidler JE, Schwartz LH, Yu KK, Adams D. Value of expert interpretation in abdominal oncologic imaging: a multicenter study (abstr). Radiology 1997; 205(P):225.
4. Wakeley CJ, Jones AM, Kabala JE, Prince D, Goddard PR. Audit of the value of double reading magnetic resonance imaging films. Br J Radiol 1995; 68:358-360.[Abstract]
5. Kalbhen CL, Yetter EM, Love L, Moncada R, Lawson TL, Albain KA. Outside film reviews of thoracic imaging studies in oncology patients (abstr). AJR 1998; 170(suppl):77.
6. Piccoli CW, Schweitzer ME, Mitchell DG, Ricci JA, Levin DC, Harford RJ. Peer-review quality assessment method for MR imaging: lower-extremity results from 31 centers (abstr). Radiology 1993; 189(P):136.
7. Bechtold RE, Chen MYM, Ott DJ, et al. Interpretation of abdominal CT: analysis of errors and their causes. J Comput Assist Tomogr 1997; 21:681-685.[Medline]
8. Bollen ECM, Goei R, Hof-Grootenboer BE, Versteege CWM, Engelshove HA, Lamers RJS. Interobserver variability and accuracy of computed tomographic assessment of nodal status in lung cancer. Ann Thorac Surg 1994; 58:158-162.[Abstract]
9. Kandis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174.[Medline]
10. Müller NL, Evans KG. Interobserver variability and accuracy of computed tomographic assessment of nodal status in lung cancer (invited commentary). Ann Thorac Surg 1994; 58:162.
11. Hilton S, Herr HW, Teitcher JB, Begg CB, Castellino RA. CT detection of retroperitoneal lymph node metastases in patients with clinical stage I testicular nonseminomatous germ cell cancer: assessment of size and distribution criteria. AJR 1997; 169:521-525.[Abstract/Free Full Text]
12. Glazer GM, Gross BH, Quint LE, Francis IR, Bookstein FL, Orringer MB. Normal mediastinal lymph nodes: number and size according to American Thoracic Society mapping. AJR 1985; 144:261-265.[Abstract/Free Full Text]

This article has been cited by other articles:

				B. Siewert, J. Sosna, A. McNamara, V. Raptopoulos, and J. B. Kruskal Quality Initiatives: Missed Lesions at Abdominal Oncologic CT: Lessons Learned from Quality Assurance RadioGraphics, May 1, 2008; 28(3): 623 - 638. [Abstract] [Full Text] [PDF]

				R. B. Ruchman, J. Jaeger, E. F. Wiggins III, S. Seinfeld, V. Thakral, S. Bolla, and S. Wallach Preliminary Radiology Resident Interpretations Versus Final Attending Radiologist Interpretations and the Impact on Patient Care in a Community Hospital Am. J. Roentgenol., September 1, 2007; 189(3): 523 - 526. [Abstract] [Full Text] [PDF]

				K. Peldschus, P. Herzog, S. A. Wood, J. I. Cheema, P. Costello, and U. J. Schoepf Computer-Aided Diagnosis as a Second Reader: Spectrum of Findings in CT Studies of the Chest Interpreted as Normal Chest, September 1, 2005; 128(3): 1517 - 1523. [Abstract] [Full Text] [PDF]

				L. Berlin Potential Legal Ramifications of Whole-Body CT Screening: Taking a Peek into Pandora's Box Am. J. Roentgenol., February 1, 2003; 180(2): 317 - 322. [Full Text] [PDF]

				L. A. Loevner, A. I. Sonners, B. J. Schulman, K. Slawek, R. S. Weber, D. I. Rosenthal, G. Moonis, and A. A. Chalian Reinterpretation of Cross-Sectional Images in Patients with Head and Neck Cancer in the Setting of a Multidisciplinary Cancer Center AJNR Am. J. Neuroradiol., November 1, 2002; 23(10): 1622 - 1626. [Abstract] [Full Text] [PDF]

				T. J. White Jr, M. J. Gollub, D. M. Panicek, A. M. Bach, and R. A. Castellino CT Reinterpretation • Dr Gollub and colleagues respond: Radiology, August 1, 1999; 212(2): 601a - 601. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Gollub, M. J. Articles by Castellino, R. A. Search for Related Content PubMed PubMed Citation Articles by Gollub, M. J. Articles by Castellino, R. A.