| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Editorial |
1 From the Departments of Radiology and Epidemiology, University of Pennsylvania, Philadelphia. Received April 11, 2002; accepted April 12. Address correspondence to the author, 719 Iron Post Rd, Moorestown, NJ 08057 (e-mail: langlotz@rad.upenn.edu).
Index terms: Radiology and radiologists, research • Radiology reporting systems
Over the past 2 decades, rapid technologic advances have enabled radiologists to acquire, transmit, process, display, and store multidimensional digital images. Despite the steady advances in the way radiologists manage images, similar technologic innovations have not yet affected how radiologists communicate with their colleagues. This discrepancy is particularly striking because the information in radiology reports is just as amenable to computer-based storage, processing, and display as pixels, voxels, and images.
In this issue of the journal, Dr Hripcsak and colleagues (1) provide a welcome early look at how information technology will shape the future of radiology reporting (1). They have developed a system that uses natural language processing (NLP) to dissect and structure meaty clinical prose into small digital morsels, each containing a unique medical concept suitable for processing by a computer. Because conventional narrative radiology reports served as early stimuli to refine and improve their NLP system, rigorous evaluations have shown that its accuracy in extracting information from radiology reports is comparable with that of human experts.
Their report details an ambitious and impressive evaluation of the NLP system on a huge database of clinical radiology reports encompassing a quarter of a million patients. The authors tasked their system with creating a semantic structure for nearly 900,000 chest radiography reports, which were dictated by radiologists at Columbia-Presbyterian Medical Center over a decade of patient care. An ingenious method was used to externally validate the contents of the structured reports. The frequency and co-occurrence of a variety of clinical conditions was computed from the structured reports and compared with preselected external benchmarks, including peer-reviewed medical literature, regional crime statistics, and financial coding results.
These comparisons demonstrate that the structured reports accurately reflected the 3:2 right-to-left ratio of lung cancer, the association of pleural effusions with other clinical conditions, and the decreasing incidence of violent crime in their catchment area. Although not externally validated, their study also produced a fascinating portrait of the semantic contents of radiology reports. For example, their Table 1 provides a unique descriptive look at the occurrence of a variety of clinical conditions in chest radiography reports that ranged from tuberculosis to rib fracture. A separate analysis quantified the radiologist’s impression that a substantial fraction of reports are normal—30% of chest radiography reports in their analysis.
The accuracy was comparable with that of expert human coders—achieving 81% sensitivity and 99% specificity, confirming the results of Hripcsak and colleagues’ (1) previous studies. Furthermore, the authors showed that their software outperformed human financial discharge coders in recognizing certain important clinical conditions, such as pneumothorax.
Taken together, their results provide a rare look at the contents of radiology reports and illustrate the feasibility of NLP methods to accurately transform databases containing narrative radiology reports into rich sources of information for radiology practices.
Remaining Open Questions
As we applaud the achievements of this study, we must also consider its limitations. As the authors themselves describe, their study was restricted to chest radiography reports and to conditions that were chosen in part because prior research had already confirmed NLP accuracy in detecting them. Evaluation of performance on more complex reports is still preliminary, but one study of cross-sectional brain imaging reports (2) showed slightly lower accuracy than with human coders.
Although the system’s reported accuracy was impressive (81% sensitivity at 99% specificity), it therefore missed 19% of salient findings. That performance may be insufficient for many clinical purposes, thereby necessitating human confirmation of NLP results prior to their use for clinical decision making.
NLP created a more comprehensive coding of reports than did discharge coders. This result, however, should be viewed in light of the primary goal of discharge coding—to code the top few diagnoses that optimize reimbursement. For example, the condition of a patient admitted for a small pneumothorax after bypass surgery might intentionally be coded with other more clinically important or more highly reimbursed diagnoses, such as coronary arterial disease, myocardial infarction, hypertension, or congestive heart failure.
Another challenge already being met by the Columbia University research team is to determine whether their system could be used to structure radiology reports using vocabularies that are currently in popular use, such as SNOMED-RT (College of American Pathologists, Northfield, Ill) and ICD-9-CM (International Classification of Diseases: 9th revision, Clinical Modification, Center for Medicare and Medicaid Services, Washington, DC), rather than a vocabulary developed specifically for their system (3). If that leap were possible, the practical utility of the structured output would increase dramatically, because it could be understood readily by electronic medical record systems, billing systems, and other clinical information systems.
Finally, more work is needed to make the colossal structured report databases produced by NLP systems more accessible to the interested radiologist. It took an experienced physician and informatician more than 4 hours to formulate, test, and apply each of the database queries for this study. Until relatively inexperienced users can readily extract information from these huge databases, the structured reports they contain will seldom be used.
Need to Improve Radiology Reporting
Many of the limitations of this research arise from neither the technology the authors have developed and applied nor their experimental design. Instead, they are due to the inherent shortcomings of narrative radiology reports themselves. There is a growing body of literature that illustrates these shortcomings. For example, radiology reports sometimes do not address the key clinical question (4), contain clinically important errors (5), are not transmitted in a timely fashion (6), or contain ambiguous terms (7). Also, radiologists may differ in the use of common terms in radiography reports, such as, "opacity," "density," "infiltrate," and "consolidation" (4). Even the most meticulous experimental design cannot determine whether an NLP system (or the physician expert against whom it is measured) has accurately gleaned the intended meaning of the radiologist who created the report.
Where can radiologists turn to increase the clarity and precision of their reports? Most vocabularies used for primary care are woefully incomplete for medical imaging (8). However, some radiology subspecialties have developed and adopted standardized vocabularies for reporting their clinical imaging results (9–11). While no controlled vocabulary is perfect, an explicitly defined term is preferable to a vague one (12). The Radiological Society of North America is responding to this challenge by supporting a pilot project called RadLex, which is intended to produce a unifying lexicon for radiology in collaboration with other professional organizations and standards bodies.
As radiologists build clearer and more precise modes of communication, the work reported by Hripcsak and colleagues (1) can serve another important purpose—to supplement the experience and wisdom of clinical experts. A structured report database describes not how experts believe radiologists should communicate but instead how experienced radiologists actually express themselves. Such descriptive data may not only reveal weaknesses in radiology reporting methods but also create benchmarks for radiology practices.
The extent to which the data produced by NLP systems can be put to real-time clinical use remains an open question. The Columbia University team has already published examples of its beneficial use in specific limited clinical settings (13). However, the terminologic imprecision currently inherent in narrative radiology reports may limit that applicability. Potential Advantages of Structured Reports Structured report databases provide new ways to measure and thereby improve the performance of radiologists. For example, these databases quantify the rate at which radiologists recommend computed tomography (CT) of the chest because of suspicious chest radiographic findings. At the same time, the databases track the rate at which CT of the chest depicts clinically important abnormalities. In the future, these measures, which serve as proxies for sensitivity and specificity, could provide individual radiologists with constructive feedback that would improve their clinical skills.
If structured report information could be captured in real time, a new realm of decision support would become available to the radiologist. For example, automated reasoning methods that have been available to other specialties for some time (14) could provide radiologists with diagnostic suggestions for unusual cases at the click of a button (15). Likewise, structured report information could automatically index and retrieve online teaching files or peer-reviewed literature relevant to specific clinical cases. Structured report information could also supplant other retrospective sources of clinical research data, such as administrative claims, which are widely used for research in other medical specialties but have substantial limitations for use in diagnostic imaging research (16).
Despite these exciting prospects for the future, very little has changed about radiology reporting since the first reports were produced more than a century ago. The study reported by Hripcsak and colleagues (1) gives us a glimpse of the future of radiology reporting, illustrates the benefits of structured reports, and highlights the need to change current radiology reporting methods.
Signs of these changes are already evident (17). A recent study (18) showed that referring physicians strongly prefer concise well-organized radiology reports. The DICOM Standards Committee, anticipating the need to store, retrieve, and transmit imaging reports with an explicit structure, recently approved a standard for the communication of structured report information. Some vendors of speech recognition systems already offer NLP technology similar to that described by Hripcsak and colleagues (1). Competing reporting products are now available that enable radiologists to structure reports themselves, thereby eliminating an important source of terminologic ambiguity. Also, the high frequency of normal reports documented in this study suggests that substantial time savings may be produced by the use of structured templates and macros.
These factors suggest that the use of NLP methods and other forms of structured radiology reporting will continue to grow, thereby revolutionizing how radiologists think about reporting. When radiology reports take on the structured form illustrated by this study, they facilitate clear communication, increase the availability of information resources, and foster clinical imaging research, thereby improving the practice of radiology. These advances in the way radiologists manipulate words, phrases, and concepts are a welcome complement to the changes radiologists have already seen in the way they manipulate pixels, voxels, and images.
FOOTNOTES
Dr Langlotz is a founder, shareholder, officer, and consultant to eDictation, a provider of software for the creation and distribution of medical imaging reports, and is a founder and shareholder of eMed Technologies, a provider of electronic image distribution and management solutions for medical imaging practices.
See also the article by Hripcsak et al in this issue.
REFERENCES
Related Article
This article has been cited by other articles:
|
|
 |
|
  B. F. Branstetter IV Basics of Imaging Informatics: Part 2 Radiology, July 1, 2007; 244(1): 78 - 84. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Reiner and E. Siegel Radiology reporting: returning to our image-centric roots. Am. J. Roentgenol., November 1, 2006; 187(5): 1151 - 1155. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Berlin Replacing traditional text radiology reports with image-centric reports: a shift from epiphany to enigma? Am. J. Roentgenol., November 1, 2006; 187(5): 1156 - 1159. [Full Text] [PDF]
|
|
|
|
|
|
R. H. Gottlieb Imaging for Whom: Patient or Physician? Am. J. Roentgenol., December 1, 2005; 185(6): 1399 - 1403. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. J. Dreyer, M. K. Kalra, M. M. Maher, A. M. Hurier, B. A. Asfaw, T. Schultz, E. F. Halpern, and J. H. Thrall Application of Recently Developed Computer Algorithm for Automatic Classification of Unstructured Radiology Reports: Validation Study Radiology, February 1, 2005; 234(2): 323 - 329. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
