| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Special Reviews |
1 From the Program for the Assessment of Radiological Technology and Departments of Radiology and Epidemiology and Biostatistics, Erasmus MC–University Medical Center Rotterdam, Dr Molewaterplein 40, 3015 GD Rotterdam, the Netherlands. From the 2003 RSNA Annual Meeting. Received April 22, 2004; revision requested June 29; revision received August 10; accepted September 15. Address correspondence to the author (e-mail: m.hunink@erasmusmc.nl).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
© RSNA, 2005
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
Milton Weinstein, one of the founders of the field of medical decision making, is quoted as having said "The decision will be made, if not actively then it will be made by default" (4); but the default decision is not always the optimal decision—to the contrary. Reactive decision making is often suboptimal. In this review, we advocate the PROACTIVE approach to decision making, in particular in the face of uncertainty and resource constraints.
PROACTIVE is a mnemonic (Table 1). The PRO part of PROACTIVE stands for defining the problem, reframing the problem from multiple perspectives, and focusing on the objective. The ACT part stands for expanding the alternatives, considering the consequences and associated chances of each alternative, and identifying the trade-offs involved. The IVE part stands for integrating the evidence and values, optimizing the value of interest, and exploring uncertainty. The PROACTIVE approach is an extended version of the PROACT approach for making choices, which was introduced by Hammond et al (5). The PROACTIVE approach was developed specifically for medical decision making but is also helpful for making personal decisions (4). Within the framework of PROACTIVE medical decision making, various formal tools can be used to qualitatively and quantitatively optimize the choices we make (Table 1).
|
|
THE PROBLEM |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
Imaging all patients would lead to an enormous use of health care resources. For example, 2 million adults visit the emergency room annually in the United States because of head trauma (8) (Fig 1). Traumatic intracranial lesions are present in 6%–9% of patients, but only 1% require neurosurgical intervention. Should we perform head CT in all these 2 million patients? Similarly, there are 800 000 cases of neck trauma annually in the United States (8) (Fig 2). Abnormal imaging findings are present in 0.9%–2.8% of these patients (10), but fewer than 1% have a cervical spinal cord injury (8). Should we perform cervical spine CT in all these patients, or would cervical spine radiography suffice? When is imaging necessary?
|
|
|
REFRAME FROM MULTIPLE PERSPECTIVES |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
For example, a patient with possible arterial injury due to trauma to an extremity (Fig 3) will want to be imaged with the best available tests and with multiple imaging modalities. The physician’s code is to "first, do no harm," which implies that the risks of intraarterial angiography will need to be carefully weighed against the risk of the patient actually having an arterial injury. Furthermore, liability and the chance of medicolegal consequences in the future if an arterial injury is missed will play a role in the physician’s decision-making process. A manager in the radiology department or hospital will be more concerned about the budget. The health care policy maker ideally takes a societal perspective and is concerned about making and keeping the health care system affordable and sustainable.
|
|
From the societal perspective, health care costs are spiraling upward: Whereas in 1985, the United States spent 10% of its gross domestic product on health care expenditures, it currently spends 14% (1–3). Furthermore, we notice an enormous variation in practice and resource utilization across regions, countries, and continents. For example, the proportion of patients with head injury who undergo head CT varies from 16% to 70% (18), and the proportion of patients with neck injury who undergo cervical spine radiography varies from 37% to 73% (19). But maybe we, as a society, want to spend more on health care than we currently do. Perhaps we are willing to pay even more than we do now to extend our lifespan and improve our quality of life. More money spent on health care, however, does not necessarily imply better health outcomes (20,21).
|
OBJECTIVES |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
|
ALTERNATIVES |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
It is always helpful to consider three main groups of options: (a) wait and see, do nothing; (b) treat immediately; and (c) obtain more information. For example, in the diagnosis of arterial injury related to trauma to an extremity (Fig 3) the cost-effectiveness analyses considered in two studies (14,15) were observation, surgical exploration, and angiography. Unfortunately, the authors of neither study considered minimally invasive or noninvasive tests such as multi–detector row CT angiography, MR angiography, or duplex US.
|
CONSEQUENCES AND CHANCES |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
A balance sheet is a table that summarizes all the information in a systematic way. For example, the important attributes (including consequences, chances, and value outcomes) of the decision problem can be listed in the columns, while the options are listed in rows. A cross tabulation results, with the collected objective evidence and subjective values for all possible options on all attributes.
A decision model is a mathematic representation of all the options, the possible consequences, the associated chances, and the outcomes. Modeling of the consequences and chances of decisions related to trauma imaging can generally be accomplished with fairly straightforward models. Markov models, which are more complicated decision models used to model recurring events over time, are rarely necessary for the evaluation of trauma imaging, because trauma concerns a one-time event. Instead, life expectancy can be modeled conditional on the age of the patient, and quality of life can be modeled conditional on the residual disability after the injury.
Decision models are particularly useful in the evaluation of diagnostic test strategies because of the myriad of possible combinations and sequences of tests and possible thresholds of test variables. The associated chances that are important in the evaluation of diagnostic test strategies include the prior probability of disease, the sensitivity and specificity of the diagnostic tests, the probability of uninterpretable test results, the mortality and morbidity related to the diagnostic tests, and the probability of therapy being successful. (Apart from the associated chances, there are also outcome values that need to be considered; these will be discussed in the section on Trade-offs.)
When appraising a decision and cost-effectiveness analysis, we need to carefully assess whether the assumptions about events and their chances are realistic and applicable. Technologic advancements are rapid, data are incomplete and messy, and the probability estimates need to be constantly adjusted to take into account the best available evidence and the situation at hand.
For example, two cost-effectiveness analyses were published in which the authors evaluated whether the performance of CT is efficient in hemodynamically stable patients with blunt chest trauma (Fig 5). Both analyses were performed before the advent of multi–detector row CT angiography (24,25). The assumptions about the sensitivity and specificity of CT—67% and 58%, respectively—in the study by Brasel and Weigelt (25) are far too low given the current experience with multi–detector row CT angiography. The assumptions we made in our analysis (24) were more optimistic—a sensitivity of 83% and a specificity of 79%—but were still too low compared with what we can expect with multi–detector row CT angiography. Brasel and Weigelt concluded that CT was not cost-effective in patients with blunt chest trauma and that aortography should be performed in all such patients. We concluded that CT was cost-effective in patients with blunt chest trauma but that the decision depended on the prior probability of aortic rupture and whether CT was required for other injuries. A more recent study (26) showed high sensitivity (100%) and cost-savings with the use of helical CT to aid in detection of thoracic aortic rupture, which suggests that multi–detector row CT angiography plays an important role in current-day practice. However, no formal cost-effectiveness analysis has been performed. This example highlights the importance of a careful appraisal of the assumptions and evidence used in a decision model and a cost-effectiveness analysis before the conclusions are accepted. Furthermore, it illustrates how different cost-effectiveness analyses on the same topic can result in different conclusions, depending on the assumptions made and the evidence used.
|
|
TRADE-OFFS |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
The QALY is a composite outcome measure that combines life expectancy and quality of life. In enumerating QALYs, we weigh each life-year with a factor that reflects the quality of that life-year (4). QALY calculations require measurement of the quality of life with a utility measurement such as a visual analogue scale, rating scale, time trade-off, or standard gamble. Utility is a global value that a patient places on his or her overall quality of life compared with life in full health. For example, the utility of life with paralysis was estimated to be 0.52, or 52% of the value of life in full health (Fig 2).
With a visual analogue scale, the respondent is asked to make a mark on a line from very poor quality of life (equivalent to death) to excellent quality of life (equivalent to life in full health). The rating scale asks the same thing but on a scale from 0, representing death, to, for example, 100, representing life in full health.
With the time trade-off method, we assess the patient’s point of indifference between living a long life with symptoms and living a shorter life without symptoms (Fig 6) (4). For example, if a patient with a life expectancy of 10 years values life with paralysis equivalent to 5 years in full health, then we would say that a year of life with paralysis is equivalent to five divided by 10, or 50% of the value of a year in full health.
|
Measures of quality of life depend strongly on who is asked, when the person is asked, and which elicitation method is used. The chosen perspective influences the choice of who should be asked. Although patients with the disease can generally be expected to know the most about their disease, they typically adjust to the circumstances and, depending on their attitude toward disability and the risk of death, generally give higher utility values than do respondents from the general public asked to consider a scenario in which they have the disease. In performing an analysis from the societal perspective, the recommendation is to use multiattribute utilities in which patients indicate the levels of various attributes and a sample from the general public has been asked to value the scenario with those levels on the attributes. Examples of multiattribute utilities are the EuroQol and the Health Utilities Index.
Another value trade-off that we commonly need to make in medical decisions is the additional expense of performing a diagnostic test in all patients versus the potentially high costs in a few patients during follow-up as a result of a missed diagnosis. For example, this trade-off plays a role in considering whether to perform CT of the cervical spine in all patients with neck trauma to prevent the loss in effectiveness and potentially high costs of care for a paraplegic in ensuing years due to a missed cervical spine injury (Fig 2).
Finally, we sometimes need to make a trade-off between additional health care costs versus non–health care cost savings. For example, MR imaging in the initial evaluation of the knee, ankle, or wrist requires additional health care resources but can reduce the "friction" costs associated with productivity losses by expediting patients’ return to work (Fig 4). Clearly, the chosen perspective of the analysis will play an important role in such trade-offs.
|
INTEGRATE |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
With a balance sheet, we get an overview of the objective evidence and subjective values for all possible options regarding all attributes. If a particular attribute is identical for all options, this attribute can be eliminated from further consideration. If an option is less desirable with regard to all attributes relative to other options, that particular option can be eliminated from further consideration. Subsequently, we need to give a relative weight to the remaining attributes in order to rank the remaining options.
With a decision model, we mathematically integrate the objective evidence and subjective values by calculating the expected value for each option. The expected value of an option is the sum across all its possible outcome values, each outcome weighted for the probability that it will occur.
The integration of risks, benefits, patient preferences, and resources commonly implies a trade-off between effectiveness and costs. Sometimes the decision is straightforward: If both effectiveness is gained and costs are saved with a new strategy, then that strategy is superior by dominance and should be implemented (Fig 7). Similarly, if effectiveness is lost and costs are incurred, then that strategy is inferior by dominance and should not be implemented. More often than not, effectiveness is gained at an additional expense, and we then need to decide whether the additional cost is justified given the effectiveness gained. In that case, we calculate the incremental cost-effectiveness ratio, which is the additional cost of changing from one strategy to the next best strategy divided by the increment in effectiveness. If the additional costs for a particular strategy per QALY gained are acceptable, in other words below society’s threshold willingness to pay for a gain of 1 QALY, we consider the strategy cost-effective (Fig 7). Sometimes a small loss in effectiveness is incurred but against large cost savings; we then need to decide whether the cost savings are so large that they justify the small loss in effectiveness. Again, the incremental cost-effectiveness ratio is important, but we now accept the new strategy as cost-effective if the cost savings are large enough—in other words if the incremental cost-effectiveness ratio is larger than the threshold willingness to pay (Fig 7).
|
|
|
VALUE |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
A commonly used value for society’s threshold willingness to pay is $50 000 per QALY, but the value ranges from $20 000 to $400 000 per QALY (28). The threshold willingness to pay is the monetary value that we assign to a QALY. It is the ceiling incremental cost-effectiveness ratio (ie, maximum amount we are willing to spend to obtain an additional QALY) or the minimum amount we believe needs to be saved if we are to forego a QALY. At a threshold willingness to pay of $50 000 per QALY, CT for neck trauma would be considered cost-effective in patients at high or moderate risk for a cervical spine injury but not in those at low risk (Fig 2, Table 2).
Incremental cost-effectiveness ratios can be difficult to interpret (Fig 7). For a given threshold willingness to pay, we can combine the effectiveness and cost outcomes into one composite outcome, the NHB, that implicitly makes the trade-off between the two. The NHB is the effectiveness (Eff) minus the cost (C) in dollars, the latter transformed to QALY equivalents by dividing by the threshold willingness to pay (WTP) in dollars per QALY, in the following equation: NHB = Eff – (C/WTP).
The NHB expresses the overall benefit in QALY equivalents, taking into account the costs and what society is willing to pay. In theory, this is equivalent to the use of the incremental cost-effectiveness ratio; in practice, however, it is easier to interpret: An NHB greater than zero implies that the strategy is cost-effective, and the higher the NHB the more cost-effective the strategy. For example, if we assume a threshold willingness to pay of $50 000 per QALY, CT for neck trauma instead of cervical spine radiography in patients at high risk for a cervical spine injury yields 4.4 NHBs per 1000 patients (Table 2). For those at moderate risk, the use of CT would still provide an NHB increase (0.9 per 1000 patients), but to a lesser degree, compared with the use of radiography. For those at low risk, there would be an NHB loss, and the use of CT would not be considered cost-effective.
What is particularly striking when we perform these types of cost-effectiveness analyses is the small net benefit that can be gained by replacing one diagnostic strategy with another. Recognize, however, that although 0.9–4.4 QALYs per 1000 patients seems small, it represents an average across all patients who present with neck trauma. Most of these patients will not benefit at all from the replacement of cervical spine radiography with CT, but there will be a small minority in whom an otherwise undetected cervical spine injury will be diagnosed and in whom paralysis can potentially be avoided for the remaining lifetime of that patient. Furthermore, we need to recognize that in our Western society we already have very sophisticated diagnostic methods, and replacing one sophisticated method with a slightly better one will not have a large impact on effectiveness or costs.
|
EXPLORE |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
The art of being wise is the art of knowing what to overlook.
William James (29)
Whenever we analyze a decision, we need to make simplifying assumptions. This applies in particular to the development of decision models, but it is also relevant to clinical studies. The key is to explore the effect of the assumptions with sensitivity analysis and to determine how different variable values affect the decision. Sensitivity analysis is a "what if?" analysis: We change the value of the variable that is uncertain across the range of plausible values (eg, 95% confidence interval) and determine how the decision changes. We can perform one-way, two-way, or n-way (multivariable) sensitivity analyses in which one, two, or multiple variables, respectively, are analyzed simultaneously. Best- and worst-case scenarios are commonly explored in which extreme values are used first to bias the results toward one strategy, and subsequent values are used thereafter to bias the results toward the competing strategy. In the analysis of new technology, it can be useful to determine target values for specific parameters that would need to be met to make the new technology cost-effective relative to current technology (30–32). For example, in the development of multi–detector row CT angiography, it was useful to determine the target sensitivity and specificity values that needed to be met.
The state-of-the-art method for exploration of the effect of uncertainty around multiple variables is probabilistic sensitivity analysis, in which Monte Carlo simulation is used (4,33). Uncertainty of all the input variables is modeled with probability distributions of their values. The model is run multiple times, and for each run the value of each input value is picked at random from its distribution. The performance of a large number of iterations yields a distribution of the results and, thus, the uncertainty around the outcome.
For example, in the evaluation of the Canadian CT prediction rule for minimal head injury (9), the investigators analyzed the cost-effectiveness of the use of the prediction rule to refer patients for CT if they were at high risk for brain injury. Their model was based on multiple input variables, and they explored the uncertainty around each of the variables in a probabilistic sensitivity analysis. They estimated that scanning only patients at high risk for brain injury would save the Canadian health care system Can$5 million, with a range from Can$1 million to Can$9 million (9). The cost savings per patient were estimated at Can$26 with a range from Can$5 to Can$43. These results indicate that even though we are uncertain of the magnitude of the cost-savings, we are confident that the prediction rule will be cost saving.
Probabilistic sensitivity analysis provides a measure of uncertainty around our outcome estimates and can be used to estimate the likelihood of making the wrong decision. But it is not only the probability of a suboptimal decision that is important to consider but also the foregone benefit due to the suboptimal decision. The value-of-information analysis, a new method in the area of cost-effectiveness analysis, combines both the likelihood of making the wrong decision and the foregone benefit of that wrong decision (34–37). The value-of-information analysis has been embraced by the National Institute For Clinical Excellence in the United Kingdom as a rational framework for the setting of research priorities in health care (38). Whereas a cost-effectiveness analysis helps us decide which strategy optimizes QALYs, costs, and NHBs, a value-of-information analysis helps us decide whether more information is necessary. It is, in fact, a logical next step after performance of a probabilistic sensitivity analysis with a second-order Monte Carlo analysis.
The expected value of perfect information (EVPI) estimates the value of obtaining information about all unknown parameters from an infinitely large sample—in other words, the value of removing all uncertainty related to the decision problem (35). EVPI is expressed in the same units as that of the cost-effectiveness analysis, preferably NHBs. By taking into account estimates of (a) the size of the population that could potentially benefit from the acquisition of more information and (b) the lifetime of a new technology, we can calculate the population EVPI. For example, Coyle et al (9) considered obtaining perfect information with respect to using the CT prediction rule for head injury and estimated the expected value to be Can$3 per patient. For an annual patient population of 2 million (Fig 1) and assuming the decision would be relevant for 10 years, this would translate to an expected value of roughly Can$60 million (without discounting).
It is also possible to calculate the EVPI for individual parameters or sets of parameters, and this is called the partial EVPI. The partial EVPI identifies which parameters have the highest informational value and can guide the choice of relevant outcome measures in a future study. For example, in the assessment of MR imaging for the initial evaluation of knee injury we estimated that approximately 90% of the expected value of information would be obtained by measuring only the days off work (Fig 4) (16).
If the EVPI is large enough, it is useful to calculate what the value would be of reducing uncertainty by obtaining information from a future study with a finite sample size, and this is called the expected value of sample information, or EVSI (39). A comparison between the EVSI and the costs of performing research tells us whether a new study is justified given the cost. The EVSI minus the cost of research gives us the expected net benefit of sampling. The optimal sample size is determined by calculating the sample size that maximizes the expected net benefit of sampling. The latter sample size calculation optimizes the use of research resources from the societal perspective.
In summary, a PROACTIVE approach to decision making can help identify (a) alternative imaging strategies; (b) the consequences in terms of risks, benefits, patient values, and costs; (c) the effect of uncertainty; and (d) the value of performing further research.
A review of published cost-effectiveness analyses of trauma imaging suggests that (a) the use of a prediction rule to select patients with minimal head injury for head CT is cost-effective; (b) screening CT of the cervical spine in patients with neck trauma is cost-effective but depends on the likelihood of a cervical spine injury; (c) angiography is cost-effective in facilitating diagnosis or exclusion of arterial injury after extremity trauma, but minimally invasive tests need to be evaluated for this indication; (d) MR imaging may be cost-effective in knee trauma, but further research is justified in this area; and (e) multi–detector row CT angiography is probably cost-effective as an alternative to angiography in cases of possible aortic rupture, but a formal cost-effectiveness analysis still needs to be performed.
|
ESSENTIALS |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
To optimize decision making, one should define the problem from multiple perspectives, focus on the objective, explore the alternatives, and consider the consequences and trade-offs involved.
imulation models play an important role in the assessment of imaging strategies by helping identify alternative strategies and integrate the best-available evidence related to risks, benefits, patient values, and costs.
Exploring the uncertainty in the evidence and assessing the value of obtaining more information can help prioritize future research and guide study design.
|
FOOTNOTES |
|---|
Abbreviations: EVPI = expected value of perfect information, NHB = net health benefit, QALY = quality-adjusted life-year
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONTHE PROBLEMREFRAME FROM MULTIPLE...OBJECTIVESALTERNATIVESCONSEQUENCES AND CHANCESTRADE-OFFSINTEGRATEVALUEEXPLORE ESSENTIALSREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  M Smits, M G M Hunink, P J Nederkoorn, H M Dekker, P E Vos, D R Kool, P A M Hofman, A Twijnstra, G G de Haan, H L J Tanghe, et al. A history of loss of consciousness or post-traumatic amnesia in minor head injury: "conditio sine qua non" or one of the risk factors? J. Neurol. Neurosurg. Psychiatry, December 1, 2007; 78(12): 1359 - 1364. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Smits, D. W. J. Dippel, G. G. de Haan, H. M. Dekker, P. E. Vos, D. R. Kool, P. J. Nederkoorn, P. A. M. Hofman, A. Twijnstra, H. L. J. Tanghe, et al. Minor Head Injury: Guidelines for the Use of CT A Multicenter Validation Study Radiology, December 1, 2007; 245(3): 831 - 838. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Smits, D. W.J. Dippel, E. W. Steyerberg, G. G. de Haan, H. M. Dekker, P. E. Vos, D. R. Kool, P. J. Nederkoorn, P. A.M. Hofman, A. Twijnstra, et al. Predicting Intracranial Traumatic Findings on Computed Tomography in Patients with Minor Head Injury: The CHIP Prediction Rule Ann Intern Med, March 20, 2007; 146(6): 397 - 405. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. M. Maher, P. A. Hodnett, and M. K. Kalra Evidence-based Practice in Radiology: Steps 3 and 4--Appraise and Apply Interventional Radiology Literature Radiology, March 1, 2007; 242(3): 658 - 670. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. C. Black Randomized Clinical Trials for Cancer Screening: Rationale and Design Considerations for Imaging Tests J. Clin. Oncol., July 10, 2006; 24(20): 3252 - 3260. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Smits, D. W. J. Dippel, G. G. de Haan, H. M. Dekker, P. E. Vos, D. R. Kool, P. J. Nederkoorn, P. A. M. Hofman, A. Twijnstra, H. L. J. Tanghe, et al. External Validation of the Canadian CT Head Rule and the New Orleans Criteria for CT Scanning in Patients With Minor Head Injury JAMA, September 28, 2005; 294(12): 1519 - 1525. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||
