Published online before print May 23, 2007, 10.1148/radiol.2441051790
(Radiology 2007;244:31-38.)
© RSNA, 2007
Evidence-based Radiology Series |
Technology Assessment in Radiology: Putting the Evidence in Evidence-based Radiology1
William Hollingworth, PhD and
Jeffrey G. Jarvik, MD, MPH
1 From the Departments of Radiology (W.H.), Pharmacy (W.H., J.G.J.), and Neurological Surgery (J.G.J.), University of Washington, Box 359960, 325 Ninth Ave, Seattle, WA 98104-2499. Received November 3, 2005; revision requested December 21; revision received January 12, 2006; final version accepted February 6.
Address correspondence to W.H. (e-mail: willh{at}u.washington.edu).
 |
ABSTRACT
|
|---|
In this review, which is part of a larger series on evidence-based practice in radiology, the relationship between technology assessment (TA) and the practice of evidence-based radiology (EBR) is discussed. TA guides researchers in the methods required to be reliable providers of unbiased and relevant evidence. Meanwhile, EBR equips radiologists with the skills needed to be discerning consumers of that evidence. Both paradigms aim to improve the effectiveness of health care spending. In this review, it is argued that EBR can be only as good as the TA on which it is based. However, TA is particularly complex in regard to diagnostic radiology because of the many links in the chain between the interim objective (to make the correct diagnosis) and the ultimate goal (to improve patient health). In this article, the development of TA in medicine in general and, more specifically, the TA hierarchy for the evaluation of diagnostic imaging are described. Some of the improvements in the pool of evidence during the past 30 years are documented, and some of the remaining tensions between TA and EBR are highlighted.
© RSNA, 2007
|
INTRODUCTION
|
|---|
The aim of evidence-based radiology (EBR) is to identify the best research evidence on a topic, use clinical skill to distill the most pertinent evidence, and combine this with knowledge about the values of individual referring clinicians and patients to select the most effective diagnostic or interventional technology (1). Nothing about this description is particularly novel or controversial; most radiologists would affirm that they informally attempt to apply all of the above in making most clinical decisions. However, the growth of the evidence-based movement in medicine since the early 1990s is grounded on a belief that mere acceptance of these sentiments is not enough and should be replaced with a more explicit method for ensuring the incorporation of evidence in practice. This article is one of a series of articles in Radiology that describes evidence-based practice and its application to clinical radiology (2–8). The EBR series will continue in the August 2007 issue of Radiology with an article by Santiago Medina and Craig Blackmore, and in the September 2007 issue with an article by Jean Raymond and Isabelle Trop.
The mantra of "evidence based" is now so indiscriminately applied in medicine that, for many, its original meaning has become lost. The early proponents of evidence-based medicine were at pains to point out that it was a bottom-up exercise, whereby individual clinicians acquired the skills necessary to assimilate the best evidence and blend it into their everyday practice. It was not initially intended to be a top-down exercise, whereby policymakers or experts used the evidence to arrive at coverage decisions or to develop clinical guidelines (9). The basic steps of bottom-up EBR are outlined in Figure 1 and are described in previous articles in this series (3–7).
The success of EBR rests heavily on three factors. First, individual radiologists must have fast access to the evidence. Great strides have been made in dragging the evidence from its traditional resting place on the shelves of the library toward a more appropriate home at the fingertips of the physician. Second, there must be valid and relevant evidence waiting to be uncovered. If every article found on a topic is no more than expert opinion, then EBR will revert to "eminence-based medicine" (10). Therefore, EBR can only be as good as the technology assessment (TA) on which it is based. Third, radiologists must believe that avoiding haphazard clinical practice is worth the time invested in the evidence-based paradigm.
EBR and TA have a common aim—to reduce the amount of medical care that is based purely on conventional wisdom or inadequate scientific evidence. In the long run, therefore, the aim of both techniques is to improve the effectiveness of the 6%–15% of gross domestic product that the world's richest nations spend on health care (11). However, EBR and TA have different starting points for this task. EBR equips radiologists with the skills needed to be discerning consumers of the evidence. Meanwhile, TA guides researchers in the methods required to be reliable providers of unbiased, relevant evidence. The tricky part, as with boring a tunnel through a mountain, is making sure the two ends meet in the middle.
The purpose of this review is to describe the development of TA in radiology, discuss ways in which TA can be made more accessible for radiologists wishing to practice EBR, and to illustrate some of the tensions between TA and EBR.
|
DEVELOPMENT OF TA IN RADIOLOGY
|
|---|
The technology in TA is defined broadly to include "drugs, devices, medical and surgical procedures used in health care, and the organizational and supportive systems within which such care is provided" (12). The United Kingdom Health Technology Assessment Programme suggests that TA should ask four fundamental questions: Does the technology work, for whom, at what cost, and how does it compare with alternatives? (13).
In most medical specialties, therapeutic technologies have a direct effect on patient health. This clear pathway between cause and effect makes TA relatively straightforward when techniques such as randomized controlled trials, cohort studies, and case-control studies are used. In diagnostic radiology, there are several additional links in the chain between the interim aim (to make the correct diagnosis) and the ultimate aim (to improve patient health). This muddies the waters both for academic radiologists conducting TA and for their clinical colleagues interpreting and implementing the evidence through EBR.
The scientific assessment of the effectiveness of medical technologies has a long history that dates back at least to the 18th-century controlled trial of six putative treatments for scurvy in British sailors, which included sea water, vinegar, and citrus fruit (14). Subsequent refinements have included the introduction of blinding (15) and randomization (16) to minimize the potential for patient and investigator bias.
High-quality medical TA has been conducted with increasing frequency since 1945. By the 1960s, randomized trial methodology had been applied to imaging to evaluate screening mammography (17). However, governmental interest in TA was formalized only in 1972 by the establishment of the U.S. Congressional Office of Technology Assessment (18). This political interest was sparked, in large part, by seemingly irreversible increases in health care expenditures and reports of medical technologies that remained untested and potentially harmful, ineffective, or inefficient. While increasing public expectations of health care and an aging population have both been cited as contributory causes of increasing health care expenditures, most scrutiny has been directed at expensive new medical technologies (19). Diagnostic radiology immediately came under the spotlight as the home of several conspicuous big-ticket technologies, including computed tomography (CT) and subsequently magnetic resonance (MR) imaging and positron emission tomography. It is not coincidental that one of the first studies published by the U.S. Congressional Office of Technology Assessment in the 1970s was an evaluation of CT scanners (20).
|
TA HIERARCHY
|
|---|
In response to this heightened attention from policymakers, researchers, on the basis of the pioneering work of Fineberg (21), have developed a framework for TA in regard to diagnostic radiology. There are several variations on this framework (22–24); all recognize the multiple links in the chain between generating an anatomically representative image and, ultimately, causing an improvement in patient health at an affordable cost. We use the terminology of Mackenzie and Dixon (25) to describe TA (Fig 2). The elements described in Figure 2 are hierarchical; therefore, a positive effect at any level generally requires a positive effect at all preceding levels. For example, a radiologic study that does not include anatomically or functionally representative images (ie, poor technical performance) will tend to result in inferior sensitivity and specificity (ie, poor diagnostic performance), continued diagnostic uncertainty (ie, little diagnostic impact), ill-judged changes in therapy (ie, poor therapeutic impact), and no net benefit to the patient (ie, no impact on health). The correlation between each successive level of the hierarchy will vary according to clinical setting; therefore, an improvement in diagnostic performance may lead to vast changes in therapy in some situations but to minor or no changes in other situations. By delineating each of these five levels, TA is used to identify any weak or missing links in the causal chain. Sometimes a missing link will be outside the domain of radiology; even a perfectly accurate diagnostic technique may not improve patient outcomes if no effective therapy is available. This fact emphasizes the need for radiologists to go beyond the question, "On the basis of the evidence, is this imaging test the most accurate?" and toward the question, "On the basis of the evidence and my clinical experience, is an accurate diagnosis likely to change management and help the patient?"
Cost-effectiveness can be calculated at each level of the hierarchy (Fig 3). Cost-effectiveness is obviously important for policymakers trying to maximize the effectiveness of finite health care resources. The role of evidence of cost-effectiveness for radiologists practicing EBR was, initially, less well defined. Indeed, one of the early criticisms of evidence-based medicine was that it encouraged clinicians to do everything possible for the individual patient without considering the wider consequences that excessive spending on one patient might have on other patients yet to be treated (26). In recent years, this rift between the goals of evidence-based medicine and that of TA has narrowed because newer guidelines have offered guidance to clinicians on how to incorporate evidence of cost-effectiveness into their decisions (1,27,28).
In many situations, implementing the most cost-effective care will be commensurate with implementing the most effective care. However, this will not always be the case; whole-body CT screening is a commonly cited recent example of diagnostic technology in which the minimal gains in life expectancy or quality of life seem to be outweighed by the cost of screening, diagnostic work-up of true- and false-positive findings, and treatment. This example demonstrates the importance of considering all levels of the TA hierarchy. The estimated cost per patient examined with whole-body CT screening is relatively modest ($900); conversely, the cost per life year gained is much higher ($151 000) because of the high cost of evaluating false-positive findings (29).
|
HOW WELL ARE WE DOING AT TA?
|
|---|
In the inaugural review of CT in 1978, the U.S. Congressional Office of Technology Assessment study revealed that "[w]ell-designed studies of efficacy of CT scanners were not conducted before widespread diffusion occurred" (20). Furthermore, only two studies had examined either the therapeutic effect or the effect on health in regard to CT scanners (30,31). The sole study (30) on patient outcomes had a weak study design and was a comparison of outcomes before and after the installation of a CT scanner; no differences in outcome were found. Despite this, the authors (20) recognized that there were numerous study results that demonstrated high diagnostic performance for CT head examinations and that CT had been more thoroughly studied than most other medical technologies at the same stage of diffusion. This lukewarm assessment of the evidence in 1978 did not greatly restrain the subsequent diffusion of CT.
The availability of evidence at all five levels of the TA hierarchy has greatly improved since 1978. At the level of diagnostic performance, the volume of research is now rarely a cause for concern; instead the emphasis is shifting to the quality of that research. For example, Smidt et al (32) evaluated the methodologic quality and reporting clarity of 124 diagnostic performance studies published in 12 journals with high impact factors during the year 2000. They report that only 41% of these studies met more than half of the quality and clarity criteria. The percentage was even lower for crucial items, such as a clear statement regarding the blinding of readers when interpreting the index test and reference standard. Inadequate blinding of readers may be associated with inflated estimates of diagnostic performance (33); therefore, this weakness in TA represents a real threat to the success of EBR.
More encouraging data suggest that the quality of diagnostic performance studies has improved over time (34), but progress has been extremely slow. Recently, the Standards for Reporting of Diagnostic Accuracy initiative has sought to quicken the pace of change. The Standards for Reporting of Diagnostic Accuracy has a 25-point checklist for researchers who are reporting the results of diagnostic performance studies; the checklist is endorsed by Radiology and many other journals (35). It is hoped that awareness of this checklist among researchers, reviewers, and editors will help to weed out flawed evidence before it reaches the public, while also improving the clarity of high-quality evidence.
Further on down the TA hierarchy, radiology research has often been accused of lacking both quantity and quality, although, once more, slow progress is being made. In 1999, Blackmore et al (36) identified 238 studies in the diagnostic and screening radiology literature on the effect on patient outcomes. Most study results were based on primary data from observational studies (59%) or randomized controlled trials (18%), and the remaining studies used secondary data already in the literature to model the effect of imaging on patient outcomes. Again, there are now established standards for the reporting of these studies of patient outcomes. Most prominently, the Consolidated Standards of Reporting Trials provides a blueprint for the description of randomized trials in Radiology and other journals (37). Similar initiatives are under way for observational studies and decision-modeling methods (38,39).
Although the quality and clarity of published evidence is improving because of the initiatives described above, readers still need to be alert to low-quality evidence. The "levels of evidence" described by the Evidence-Based Radiology Working Group (40) are designed to help readers quickly identify and discard low-quality evidence. Evidence is categorized into five broad levels of validity. For diagnostic accuracy, the top rung of the evidence ladder comprises controlled case series of diagnostic tests in consecutive patients in which all patients undergo the index test and the reference standard test. Isolated case reports are at the lowest end of the evidence ladder. Knowledge of these evidence levels will enable radiologists to base practice on the best evidence available and also increase awareness of the common flaws in the literature.
Despite improvements in radiology outcomes research, the evidence is still heavily skewed toward studies on evaluation of technical and diagnostic performance. This distribution is inevitable because these studies are cheaper, quicker, and easier to perform. The distribution is also desirable. The controlled trials needed to evaluate levels 3–5 of the TA hierarchy are costly enterprises. Therefore, they should be reserved for imaging technology in which the cost is high or the ability to improve patient health is questionable. We would not want to waste money further evaluating an imaging test known to have poor diagnostic performance.
Very infrequently, there will also be cases in which the new technique represents such a quantum advance in diagnostic performance or safety and is clearly associated with an effective therapy that further evaluation is not needed. The use of head CT for patients with severe head trauma may be one such example. When CT scanners were introduced, there was never a trial with random assignment of patients with severe head trauma to CT versus usual care (which at that time would have been cerebral angiography or simply watchful waiting). The anatomic definition and diagnostic performance yielded by CT were obviously so much better than those of alternative strategies that equipoise (genuine uncertainty about the efficacy of a medical technology) was never present. Just as important, there was an effective treatment for patients with subdural and epidural hematomas—neurosurgical evacuation. These exceptions are rare, and claims that a new technology does not need evaluation because it is obviously cost-effective should not be taken at face value. The history of medicine contains many examples of interventions whose efficacy was thought to be self-evident until they were finally proved to be ineffective or harmful (41). Therefore, in general, new imaging modalities and interventional procedures should be viewed with a degree of healthy skepticism to preserve equipoise until evidence dictates otherwise.
|
LINKING TA AND EBR
|
|---|
At some levels, there is already a convenient interface between TA and EBR. For diagnostic performance studies, there is a well-established user guide, "How to Use an Article about a Diagnostic Test" (42,43), designed to help the clinician with the third step of the EBR process, critical appraisal of the evidence (Fig 1). The guide includes tips for determining answers to questions such as: Are the results of the study valid? What are the results of the study? Will the results help me in caring for my patients? Similar user guides, which look at the effect of diagnostic imaging on patient health, are also available for randomized controlled trials (44,45). Other user guides on screening, prognosis, prediction rules, and clinical practice guidelines (46–50) are also relevant to radiologists in many situations, although they are less closely tied to the TA hierarchy in Figure 2. All user guides help ensure that the radiologist finds his or her way to the best evidence. To return to our previous analogy, the guides help make sure that the two ends of the tunnel meet in the middle.
The need for user guides suggests that, in the past, these critical appraisal skills have not been ingrained in the training of junior radiologists. There are signs that this is changing. In the United States, the Accreditation Council for Graduate Medical Education focus on general competencies, such as practice-based learning, has emphasized the importance of critical appraisal and assimilation of evidence in the training of residents (51).
To illustrate the interplay between the TA hierarchy and EBR, we consider a brief clinical example. A radiologist is asked by a clinical colleague, "Does MR spectroscopy increase diagnostic performance beyond that achievable with conventional MR imaging sequences in patients suspected of having brain tumors?" The radiologist searches the Cochrane Library on this topic and identifies one systematic review published by the Agency for Healthcare Research and Quality in 2003 (52). This review identified 96 articles on this topic, of which 85 were considered to be reports of studies on the technical performance level of the TA hierarchy. Results of most of these technical performance studies report encouraging correlations between MR spectroscopy, histologic findings, and survival. The abundance of technologic performance studies suggests that MR spectroscopy has reached a level of technologic maturity.
However, these studies provide no estimate of diagnostic performance. The radiologist notices that of the remaining 11 studies, only one (53) compared the incremental diagnostic performance of MR spectroscopy and MR imaging with that of MR imaging alone. This study involved a consecutive series of 176 patients with focal intracranial mass lesions at presentation. All test data were interpreted by neuroradiologists who were blinded to the reference standard diagnosis. Most, but not all, patients had histologic findings available within 10 days of MR spectroscopy. In the remaining patients, the final diagnosis was established with radiologic follow-up. Because this is just one single-center study and because of the inconsistency in the application of the reference standard, the radiologist is cautious, realizing that this study is well below the level of ideal evidence (40). However, the article is the best evidence available, and it does suggest a moderate, but statistically significant, increase in diagnostic performance at MR spectroscopy combined with MR imaging (124 [70%] of 176 diagnoses correct) compared with that at MR imaging alone (97 [55%] of 176 diagnoses correct) (53). After consideration of the applicability of these findings to his or her own setting, the radiologist might choose to incorporate MR spectroscopic sequences in the diagnostic work-up of carefully selected patients and review this decision as more evidence becomes available.
However, knowledge of the TA hierarchy should also encourage the radiologist to consider broader and potentially more important questions. For example, "Does MR spectroscopy increase diagnostic confidence enough to reduce the need for biopsy (diagnostic impact)?" "Does MR spectroscopy better delineate tumor borders, thereby changing the surgical or radiation therapy target volume (therapeutic impact)?" "Would these changes result in any improvement in mortality or morbidity for the patient (impact on health)?" At this point, in this clinical example, the evidence becomes extremely sparse. The systematic review identified only three small uncontrolled studies (54–56) on estimation of the potential for MR spectroscopy to affect biopsy and therapy and identified no studies on measurement of patient quality of life or survival or cost-effectiveness. There is an opportunity here for positive feedback between EBR and TA. Having completed the EBR process and identified the limitations of the existing evidence, the radiologist is in a good position to suggest or initiate research that will extend the current boundaries of evidence.
|
TENSIONS BETWEEN TA AND EBR
|
|---|
Although TA and EBR share the same goal, there are a number of differences in emphasis between the two, which may lead to tensions (Table). Much of the friction between TA and EBR arises through a mismatch in timing. Decisions need to be made immediately, whereas good evidence takes time to collect. The natural inclination for many researchers conducting TA is to invest the time necessary to get a definitive answer. However, this can leave decision makers in limbo for a considerable period of time. A review of 59 trials in the radiation oncology literature found that the average time to complete the trial and publish the results was about 11 years (57). This reemphasizes the importance of rapid studies at higher levels of the TA hierarchy whose results attest to the potential value of an imaging technology (58). The problem of trying to practice evidence-based medicine in the face of insufficient evidence is not unique to radiology (59). Individual clinicians and policymakers need to be bold enough to make interim decisions on the basis of a mixture of clinical experience and the available, but imperfect, evidence. They also need to be flexible enough to reverse those decisions if they are subsequently proved wrong.
Timing is a particular problem for rapidly evolving technology. Buxton's law of TA states that "it is always too early to evaluate until, suddenly, it is too late" (60). Research showing poor performance of a nascent imaging technique will always be criticized for being conducted too early, before technical issues have been ironed out. Whereas research conducted on a mature technology will often be considered unethical because equipoise has vanished. There is no simple solution to this problem.
TAs provide information that is used by a number of stakeholders, including individual radiologists, policymakers, and, increasingly, patients. This may also lead to tension; in particular, the fear among radiologists that TA has been co-opted by policymakers to justify cost cutting. The use of MR spectroscopy provides an interesting example of a situation in which decision makers may reach opposite conclusions on the basis of the same TAs. The Centers for Medicare and Medicaid Services, at the request of the American College of Radiology, reviewed their coverage policy for MR spectroscopy in 2004. On the basis of the same evidence briefly described above, the Centers for Medicare and Medicaid Services concluded that "the evidence is not adequate to conclude that [MR spectroscopy] is reasonable and necessary ... for use in the diagnosis of brain tumors" (61) and continued their national policy of noncoverage. Results of a recent study (62) have shown that the Centers for Medicare and Medicaid Services reached a noncoverage determination for about one-third of technologies that it reviewed between 1999 and 2003 and that these decisions were directly associated with the subjective quality of the evidence.
In some cases, policymakers and individual clinicians will have different interpretations of the evidence on a diagnostic technology; such differences may occur with increasing frequency as policymakers intensify the scrutiny of health care spending. In general, policymakers will demand a greater weight of evidence before committing resources to a new diagnostic technology, knowing that it is easier to withhold coverage initially than rescind coverage after widespread adoption.
One partial solution to this problem is for greater collaboration between the health care payers and professional societies to fund multicenter TAs that provide the information that both parties seek. The American College of Radiology Imaging Network is one example of collaboration between the National Cancer Institute and the College to provide the resources necessary to conduct large multi-institutional trials of imaging technology (63). Another novel example of such cooperation is the National Emphysema Treatment Trial, where the Centers for Medicare and Medicaid Services covered lung volume reduction surgery only if a patient was entered into the randomized trial. The results of the trial were used as a basis for the final coverage decision (64).
|
BRINGING TA AND EBR CLOSER TOGETHER
|
|---|
To be of most use to clinicians, TA needs to be timely, accessible, relevant, and valid (65). Designing the study, applying for grant sponsorship, hiring research staff, obtaining ethical approval, recruiting patients, measuring outcomes, analyzing data, and publishing results is a multiyear process. This is particularly true for studies on the effect of radiology on patient health, which may require long-term follow-up. This is an issue that shows few signs of being resolved in the near future; indeed, some aspects of the research process are becoming more protracted. Therefore, it is important that radiologists regularly revisit the evidence underlying their practice. Services such as the "MyNCBI" feature of the National Library of Medicine, which can be set up to automatically e-mail results of relevant trials or systematic reviews as they are published, are invaluable in the battle to keep abreast of new information without being overwhelmed by it.
The accessibility of current evidence has improved immeasurably in recent years. Compact, regularly updated, synopses and syntheses of the evidence on selected clinical questions can be readily accessed with Clinical Evidence (http://www.clinicalevidence.com), Up to Date (http://www.uptodate.com), the Cochrane Library (http://www.cochrane.org/reviews/clibintro.htm), or Evidence Based Medicine (http://ebm.bmjjournals.com) and its sister journals (66). Unfortunately, radiology has lagged behind other clinical specialties in providing a journal forum for the publication of literature synopses. Therefore, it is often still necessary to use the National Library of Medicine PubMed utility to sift through the more than 600 000 articles published in the medical literature annually. Evidence-based practice search strategies have been discussed earlier in this series (3).
Ensuring that TA is relevant for clinical practice is a continuing challenge. There are several steps that those conducting TA can take to ensure that the evidence they create is relevant. First, research should aim to recruit the full range of patients likely to undergo imaging in routine clinical practice, not just a subset of highly selected patients who might benefit most. Second, where funding permits, studies should include multiple imaging centers, thereby increasing the generalizability of results and enabling large sample sizes for statistically precise results. Finally, studies should evaluate predefined subgroups to try to identify those subsets of patients in which imaging has the greatest and least benefit. This pragmatic approach to research will optimize its usefulness to policymakers and clinicians.
Authors, editors, reviewers, and journal readers all have a role in ensuring that published TA is scientifically valid and clearly written. With the publication of the Standards for Reporting of Diagnostic Accuracy and Consolidated Standards of Reporting Trials guidelines, this is one area of EBR that should improve markedly during the next few years.
|
CONCLUSION
|
|---|
The quantity and accessibility of medical evidence have increased dramatically during the past 2 decades. This, coupled with the gradual increase in methodologic quality of TA, provides unprecedented opportunities to promote effective and efficient clinical radiology. However, there are many remaining obstacles. Researchers and funding agencies need to find better ways of publishing relevant evidence faster. Furthermore, EBR requires continued commitment from current radiologists to learn and implement critical appraisal in their own practice and to teach these skills to the next generation of radiologists (8).
|
FOOTNOTES
|
|---|
Abbreviations: EBR = evidence-based radiology • TA = technology assessment
Authors stated no financial relationship to disclose.
|
References
|
|---|
- Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 3rd ed. Edinburgh, Scotland: Elsevier Churchill Livingstone, 2005.
- Malone DE. Evidence-based practice in radiology: an introduction to the series. Radiology 2007;242(1):12–14.[Free Full Text]
- Staunton M. Evidence-based radiology: steps 1 and 2—asking answerable questions and searching for evidence. Radiology 2007;242(1):23–31.[Abstract/Free Full Text]
- Dodd JD. Evidence-based practice in radiology: steps 3 and 4—appraise and apply diagnostic radiology literature. Radiology 2007;242(2):342–354.[Abstract/Free Full Text]
- Maher MM, Hodnett PA, Kalra MK. Evidence-based practice in radiology: steps 3 and 4—appraise and apply interventional radiology literature. Radiology 2007;242(3):658–670.[Abstract/Free Full Text]
- Halligan S, Altman DG. Evidence-based practice in radiology: steps 3 and 4—appraise and apply systematic reviews and meta-analyses. Radiology 2007;243(1):13–27.[Abstract/Free Full Text]
- Malone DE, Staunton M. Evidence-based practice in radiology: step 5 (evaluate)—caveats and common questions. Radiology 2007;243(2):319–328.[Abstract/Free Full Text]
- van Beek EJR, Malone DE. Evidence-based practice in radiology education: why and how should we teach it? Radiology 2007;243(3):633–640.[Abstract/Free Full Text]
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996;312(7023):71–72.[Free Full Text]
- Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999;319(7225):1618.[Free Full Text]
- Organisation for Economic Co-operation and Development. OECD health data 2005. Paris, France: Organisation for Economic Co-operation and Development, 2005.
- Eisenberg JM, Zarin D. Health technology assessment in the United States: past, present, and future. Int J Technol Assess Health Care 2002;18(2):192–198.[CrossRef][Medline]
- White SJ, Ashby D, Brown PJ. An introduction to statistical methods for health technology assessment. Health Technol Assess 2000;4(8):i–iv, 1–59.[Medline]
- Enkin MW. Foreward. In: Randomised controlled trials. London, England: BMJ Books, 1998; vii–x.
- Schulz KF, Chalmers I, Altman DG. The landscape and lexicon of blinding in randomized trials. Ann Intern Med 2002;136(3):254–259.[Free Full Text]
- Doll R. Controlled trials: the 1948 watershed. BMJ 1998;317(7167):1217–1220.[Free Full Text]
- Shapiro S, Strax P, Venet L. Evaluation of periodic breast cancer screening with mammography: methodology and early observations. JAMA 1966;195(9):731–738.[Abstract/Free Full Text]
- Banta D. The development of health technology assessment. Health Policy 2003;63(2):121–132.[CrossRef][Medline]
- Bodenheimer T. High and rising health care costs. II. Technologic innovation. Ann Intern Med 2005;142(11):932–937.[Abstract/Free Full Text]
- Office of Technology Assessment. Policy implications of the computed tomography (CT) scanner. Washington, DC: Office of Technology Assessment, 1978.
- Fineberg HV. Evaluation of computed tomography: achievement and challenge. AJR Am J Roentgenol 1978;131(1):1–4.[Medline]
- Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991;11(2):88–94.[Abstract/Free Full Text]
- Thornbury JR, Eugene W. Caldwell lecture: clinical efficacy of diagnostic imaging—love it or leave it. AJR Am J Roentgenol 1994;162(1):1–8.[Abstract/Free Full Text]
- Thornbury JR. Intermediate outcomes: diagnostic and therapeutic impact. Acad Radiol 1999;6(suppl 1):S58–S65.[CrossRef][Medline]
- Mackenzie R, Dixon AK. Measuring the effects of imaging: an evaluative framework. Clin Radiol 1995;50(8):513–518.[CrossRef][Medline]
- Maynard A. Evidence-based medicine: an incomplete method for informing treatment choices. Lancet 1997;349(9045):126–128.[CrossRef][Medline]
- Drummond MF, Richardson WS, O'Brien BJ, Levine M, Heyland D. Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1997;277(19):1552–1557.[Abstract/Free Full Text]
- O'Brien BJ, Heyland D, Richardson WS, Levine M, Drummond MF. Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA 1997;277(22):1802–1806.[Abstract/Free Full Text]
- Beinfeld MT, Wittenberg E, Gazelle GS. Cost-effectiveness of whole-body CT screening. Radiology 2005;234(2):415–422.[Abstract/Free Full Text]
- Ambrose J, Gooding MR, Uttley D. E.M.I. scan in the management of head injuries. Lancet 1976;1(7964):847–848.[Medline]
- Fineberg HV, Bauman R, Sosman M. Computerized cranial tomography: effect on diagnostic and therapeutic plans. JAMA 1977;238(3):224–227.[Abstract/Free Full Text]
- Smidt N, Rutjes AW, van der Windt DA, et al. Quality of reporting of diagnostic accuracy studies. Radiology 2005;235(2):347–353.[Abstract/Free Full Text]
- Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999;282(11):1061–1066.[Abstract/Free Full Text]
- Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research: getting better but still not good. JAMA 1995;274(8):645–651.[Abstract/Free Full Text]
- Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Radiology 2003;226(1):24–28.[Abstract/Free Full Text]
- Blackmore CC, Black WC, Jarvik JG, Langlotz CP. A critical synopsis of the diagnostic and screening radiology outcomes literature. Acad Radiol 1999;6(suppl 1):S8–S18.[CrossRef][Medline]
- Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357(9263):1191–1194.[CrossRef][Medline]
- Des Jarlais DC, Lyles C, Crepaz N. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 2004;94(3):361–366.[Abstract/Free Full Text]
- Philips Z, Ginnelly L, Sculpher M, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess 2004;8(36):iii–iv, ix–xi, 1–158.[Medline]
- Evidence-Based Radiology Working Group. Evidence-based radiology: a new approach to the practice of radiology. Radiology 2001;220(3):566–575.[Abstract/Free Full Text]
- Doust J, Del Mar C. Why do doctors use treatments that do not work? BMJ 2004;328(7438):474–475.[Free Full Text]
- Jaeschke R, Guyatt G, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1994;271(5):389–391.[Abstract/Free Full Text]
- Jaeschke R, Guyatt GH, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA 1994;271(9):703–707.[Abstract/Free Full Text]
- Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. JAMA 1994;271(1):59–63.[Abstract/Free Full Text]
- Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1993;270(21):2598–2601.[Abstract/Free Full Text]
- Barratt A, Irwig L, Glasziou P, et al. Users' guides to the medical literature. XVII. How to use guidelines and recommendations about screening. Evidence-Based Medicine Working Group. JAMA 1999;281(21):2029–2034.[Free Full Text]
- Laupacis A, Wells G, Richardson WS, Tugwell P. Users' guides to the medical literature. V. How to use an article about prognosis. Evidence-Based Medicine Working Group. JAMA 1994;272(3):234–237.[Abstract/Free Full Text]
- McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users' guides to the medical literature. XXII. How to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA 2000;284(1):79–84.[Abstract/Free Full Text]
- Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users' guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? Evidence-Based Medicine Working Group. JAMA 1995;274(7):570–574.[Abstract/Free Full Text]
- Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt G. Users' guides to the medical literature. VIII. How to use clinical practice guidelines. B. What are the recommendations and will they help you in caring for your patients? Evidence-Based Medicine Working Group. JAMA 1995;274(20):1630–1632.[Abstract/Free Full Text]
- Collins J, de Christenson MR, Gray L, et al. General competencies in radiology residency training: definitions, skills, education and assessment. Acad Radiol 2002;9(6):721–726.[CrossRef][Medline]
- Jordan HS, Bert RB, Chew P, Kupelnick B, Lau J. Magnetic resonance spectroscopy for brain tumors. Rockville, Md: Agency for Healthcare Research and Quality, 2003; 109.
- Moller-Hartmann W, Herminghaus S, Krings T, et al. Clinical application of proton magnetic resonance spectroscopy in the diagnosis of intracranial mass lesions. Neuroradiology 2002;44(5):371–381.[CrossRef][Medline]
- Adamson AJ, Rand SD, Prost RW, Kim TA, Schultz C, Haughton VM. Focal brain lesions: effect of single-voxel proton MR spectroscopic findings on treatment decisions. Radiology 1998;209(1):73–78.[Abstract/Free Full Text]
- Hall WA, Martin A, Liu H, Truwit CL. Improving diagnostic yield in brain biopsy: coupling spectroscopic targeting with real-time needle placement. J Magn Reson Imaging 2001;13(1):12–15.[CrossRef][Medline]
- Lin A, Bluml S, Mamelak AN. Efficacy of proton magnetic resonance spectroscopy in clinical decision making for patients with suspected malignant brain tumors. J Neurooncol 1999;45(1):69–81.[CrossRef][Medline]
- Soares HP, Kumar A, Daniels S, et al. Evaluation of new treatments in radiation oncology: are they better than standard treatments? JAMA 2005;293(8):970–978.[Abstract/Free Full Text]
- Sunshine JH, McNeil BJ. Rapid method for rigorous assessment of radiologic imaging technologies. Radiology 1997;202(2):549–557.[Abstract/Free Full Text]
- Diringer MN. Evidence-based medicine: what do you do when there's no evidence? Crit Care Med 2003;31(2):659–660.[CrossRef][Medline]
- Buxton MJ. Problems in the economic appraisal of new health technology. In: Drummond MF, ed. Economic appraisal of health technology in the European Community. Oxford, England: Oxford Medical Publications, 1987.
- Centers for Medicare and Medicaid Services. National coverage decision for magnetic resonance spectroscopy. Baltimore, Md: Centers for Medicare and Medicaid Services, 2004.
- Neumann PJ, Divi N, Beinfeld MT, et al. Medicare's national coverage decisions, 1999–2003: quality of evidence and review times. Health Aff (Millwood) 2005;24(1):243–254.[Abstract/Free Full Text]
- Hillman BJ. New ACRIN fellowships in imaging clinical trials: open to all radiologists. Radiology 2002;225(3):633.[Free Full Text]
- Ramsey SD, Sullivan SD. Evidence, economics, and emphysema: Medicare's long journey with lung volume reduction surgery. Health Aff (Millwood) 2005;24(1):55–66.[Abstract/Free Full Text]
- Chantler C. Health-care technology assessment: a clinical perspective. Int J Technol Assess Health Care 2004;20(1):87–91.[Medline]
- Haynes RB. Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence. Evid Based Ment Health 2001;4(2):37–39.[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
M. D. Mitchell
Technology Assessment Is Performed by Payers, Too
Radiology,
March 1, 2008;
246(3):
984 - 985.
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. S. Medina and C. C. Blackmore
Evidence-based Radiology: Review and Dissemination
Radiology,
August 1, 2007;
244(2):
331 - 336.
[Abstract]
[Full Text]
[PDF]
|
|
|
TOP
ABSTRACT
INTRODUCTION
