DOI: 10.1148/radiol.2471070745
(Radiology 2008;247:213-219.)
© RSNA, 2008
Impact of the National Institutes of Health on Radiology Research1
Michael W. Itagaki, MD, MBA
1 From the Department of Radiological Sciences, David Geffen School of Medicine at UCLA, 10833 Le Conte Ave, Los Angeles, CA 90095. From the 2006 RSNA Annual Meeting. Received April 27, 2007; revision requested June 18; revision received July 15; accepted August 16; final version accepted September 25.
Address correspondence to the author (e-mail: mitagaki{at}mednet.ucla.edu).
 |
ABSTRACT
|
|---|
Purpose: To retrospectively quantify the characteristics of published radiology research sponsored by the National Institutes of Health (NIH) and to retrospectively characterize the contribution of the individual NIH institutes.
Materials and Methods: The National Library of Medicine's PubMed database was searched for all articles published by U.S. radiology departments from 1996 to 2005. Methodology and NIH grant support were recorded for each article. Data were analyzed with linear regression curve estimation, with P 
.05 indicating a significant difference.
Results: Worldwide, 76 838 articles were identified, with 30 156 (39.25%) originating from the United States. The NIH funded 28.36% of clinical trials and 18.58% of U.S. articles overall. The proportion of funded articles increased annually by 1.1% (P < .001) overall, by 3.5% for clinical trials (P < .001), and by 5.2% for multicenter trials (P < .001). By 2005, 54.9% of all clinical trials and 55.6% of multicenter trials were NIH funded. The National Cancer Institute sponsored the most articles (2505 articles; growth, 13.5 articles per year; P = .003), followed by the National Institute of Neurological Disorders and Stroke (1030 articles; growth, 4.7 articles per year; P = .012). The newest NIH institute, the National Institute of Biomedical Imaging and Bioengineering, had the most rapid growth (320 articles; growth, 43.7 articles per year; P = .01).
Conclusion: The NIH funded less than one-fifth of all U.S. radiology articles, but the proportion of funded articles is increasing, especially for clinical trials, where in 2005 more than half received funding; cancer and neurologic disease research is relatively well funded.
© RSNA, 2008
|
INTRODUCTION
|
|---|
The National Institutes of Health (NIH) play a critical role in promoting and funding research in radiology. NIH funding level is the single greatest predictor of U.S. radiology department publication output because funding and publications are strongly correlated. Approximately 86% of the number of publications from U.S. academic departments can be predicted on the basis of their NIH funding levels (1). Because NIH funding is such a strong predictor, understanding funding characteristics and trends in radiology is important from both research and policy standpoints. This is especially so because recent NIH budgets have stagnated, future budget cuts are expected, and grant application success rates are at record low levels (2–4). Grants awarded to radiology departments are published annually by the NIH (5), but how frequently those grants translate into published research is poorly understood. Whether NIH grants are predictive because they directly generate publications or are merely correlated with some other factor that generates publications is unknown.
It may seem intuitive that NIH grants directly generate publications, but this effect is not proved. The relationship may in fact work in reverse, with successful research and publication subsequently attracting grants. The proportion of NIH-funded radiology articles could clarify the relationship between funding and publications. If a large proportion of U.S. articles are NIH funded, then NIH funding is likely directly causative. If the proportion is small, then NIH funding level is likely not causative, and other elements are at play to explain the strong correlation. For example, the NIH may fund a few key projects, which subsequently stimulate additional, unrelated research. Alternatively, NIH funding may be more of a reward given to productive departments that would be successful regardless of funding level.
The impact of the NIH on radiology research has important policy implications, because a causal relationship would suggest that U.S. radiology research is sensitive to swings in federal grant appropriations, while a noncausal relationship would suggest a more resilient relationship. Once the proportion of NIH-funded articles has been established, a reasonable next step would be to investigate any changes over time. Radiology research is constantly evolving, and any change in the funding proportion would signify a shift in the strength of the relationship between the NIH and research output. Furthermore, understanding the methodologic composition of NIH-funded articles, such as the proportion published as clinical trials, meta-analyses, or general research articles, would indicate where federal funding priorities lie in terms of research methodologies and would provide guidance for investigators seeking grants. The impact of individual NIH agencies, each with its own research agenda, has never been studied with respect to radiology, to my knowledge. Information on funding levels from individual agencies would provide an indication of the organ systems and diseases that have had the most success in attracting federal research dollars and resulting in successful publication.
Thus, the purpose of my study was to retrospectively quantify the characteristics of published radiology research sponsored by the NIH and to retrospectively characterize the contribution of the individual NIH institutes.
|
MATERIALS AND METHODS
|
|---|
Article Search
This was a retrospective analysis of publicly available data. The National Library of Medicine's PubMed database was queried by the author for all indexed articles published between and including 1996 and 2005 that contained the keyword "Radiolog" in the author affiliation field—the citation field that describes the first author's department and institution. Multispecialty articles in which a radiologist had a supportive role and was not first author were thus excluded. The selection algorithm was therefore conservative, excluding some legitimate radiology articles in which the first author was a nonradiologist to ensure that selected articles clearly qualified as radiology.
Data Collection
The matching citation data were downloaded and reconstructed in a local research computer for further analysis by the author. Each article's descriptor fields, as specified by the National Library of Medicine, were preserved, including the article publication date, author affiliation, and article methodology. Methodology categories included journal articles of all types: case reports; review articles; meta-analyses; multicenter studies; general clinical trials; and controlled, randomized-controlled, phase I, phase II, phase III, and phase IV clinical trials. In addition, a category of multicenter trial was created for articles that were both clinical trials and multicenter studies, and a category of general research article was created for articles that were classified as journal articles but belonged to no other article type. The National Library of Medicine does not have a specific category for systematic reviews and instead categorizes them as meta-analyses or review articles, as appropriate. PubMed indexing of NIH funding is standardized, and the NIH institute and grant number of funded articles was recorded. No standard mechanism exists for reporting or indexing grants from industrial or private sources; therefore, only grant data from NIH sources was included.
Articles originating from the United States were identified by statement of known U.S. institution, city, or state or the phrase "USA" in the author affiliation field. A similar method was used to assign articles to other countries. Although non-U.S. articles were not directly used in this analysis, country assignment was made anyway for completeness. In instances where the author's country of origin was ambiguous or absent, the article was not assigned to any country. As a check for accuracy of the selection algorithms, 500 randomly selected articles identified as originating from U.S. radiology departments were manually reviewed by the author. Fourteen radiation oncology (2.8%), three veterinary medicine (0.6%), and two non-U.S. radiology (0.4%) articles and one maxillofacial surgery (0.2%) article were improperly categorized as U.S. radiology articles because of unusual department names that met the keyword selection criteria. In 13 instances (2.6%), the first author had multiple department affiliations, including a radiology affiliation, and the article was properly considered to be a radiology article. The keyword selection algorithms properly identified U.S. radiology articles 95.8% of the time.
Statistical Analysis
Statistical analysis was performed by using software (SPSS for Windows, version 11.5; SPSS, Chicago, Ill). Trends in the proportion of NIH-sponsored articles overall and for each article type and number of sponsored articles from each NIH institute were examined. Ordinary least-squares linear regression analysis was used to test for a nonzero slope over time, with a threshold for significance of P .05.
|
RESULTS
|
|---|
Articles Identified
A total of 76 838 articles from 2673 distinct journals were identified as originating from radiology departments worldwide during the study years. Of those, 30 156 articles (39.25% of the world total) from 1582 journals originated from U.S. departments. Of the total world articles, 5153 (6.71%) were not assignable to any country, and the remainder were assigned to countries other than the United States.
Of U.S. articles, 30 059 (99.68%) qualified as journal articles, the difference being composed of nonjournal articles such as index articles. Of U.S. journal articles, 5586 (18.58%) received NIH grant support. The proportion of published articles that received NIH funding varied tremendously by article type. The percentage of clinical trials, multicenter studies, and general research articles receiving funding was 28.4%, 23.7%, and 25.5%, respectively. The percentage of funded case reports and review articles was low, at 1.5% and 7.3%, respectively. Other than general research articles, case reports and review articles were the most common article type in the radiology literature (Table 1).
Type of Article and NIH Funding
The total number of U.S. radiology articles was mostly stable during the 10-year period. Data from the NIH show that radiology funding increased during this same period (Fig 1). Publications for all journal articles, meta-analyses, multicenter studies (including multicenter trials), clinical trials (all types), controlled clinical trials, randomized-controlled clinical trials, multicenter trials, phase I clinical trials, and phase II clinical trials had no detectable growth. Annual publications decreased for general research articles (growth, –45.5 articles per year; P = .016), case reports or case series (growth, –26.2 articles per year; P = .001), and phase III clinical trials (growth, –0.74 articles per year; P = .004). Positive growth was found for review articles (growth, 12.6 articles per year; P = .019).
View larger version (17K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1: Graph of total published U.S. radiology articles and extramural NIH funding to U.S. radiology departments from 1996 to 2005. While NIH funding has risen steadily, published articles have been stable (P = .13 for trend). Funding figures are inflation adjusted to 2005 dollars (6). NIH-funding data are from the NIH's Office of Extramural Research (5).
|
|
While the number of U.S. radiology articles decreased or remained stable overall, the proportion of NIH-funded articles increased significantly for nine article categories: all journal articles, general research articles, review articles, multicenter studies, clinical trials (all types), and clinical trials that were controlled, randomized-controlled, multicenter, and phase II. No statistically significant growth in the proportion of NIH-funded articles was detected for phase I clinical trials or case reports or case series. Too few articles were published to enable trend analysis to be performed for NIH-funded meta-analyses or phase III or phase IV clinical trials (Table 1, Fig 2).
View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2: Graph of proportion of NIH-funded U.S. radiology articles from 1996 to 2005, according to selected methodologies. While proportion of NIH-funded U.S. radiology articles has been increasing overall, the increase is particularly strong for articles with rigorous methodologies, such as clinical trials. All methodologies shown had statistically significant positive trends.
|
|
NIH Institute Funding
When individual NIH institutes were examined, the National Cancer Institute (NCI) sponsored the greatest number of radiology articles: 2505 during the 10-year study period. The National Institute of Neurological Disorders and Stroke (NINDS), the National Center for Research Resources, and the National Heart, Lung, and Blood Institute followed with 1030, 915, and 914 articles, respectively. Twelve institutes funded more than 100 radiology articles (Table 2).
View this table:
[in this window]
[in a new window]
|
Table 2. NIH-Funded Radiology Articles according to Individual NIH Institute or Agency and 10-Year Growth Trends, 1996–2005
|
|
The National Institute of Biomedical Imaging and Bioengineering (NIBIB) was particularly notable. The NIBIB was the sixth-largest sponsoring NIH institute during the entire 10-year period; it sponsored 320 published articles. The NIBIB is the newest NIH institute, was established on December 29, 2000, and did not receive its first grant appropriation until 2002 (7). In 2002, the first seven NIBIB-sponsored radiology articles were published. This number increased rapidly, with 40 articles published in 2003, 102 in 2004, and 171 in 2005. In 2005, the last year of this study, the NIBIB was the second-largest sponsor of published radiology research, being surpassed only by the NCI, which had 341 articles to its credit. In this same year, NIBIB funded 12 published clinical trials (17.6% of the 68 total trials in 2005)—third behind the NCI (24 trials, 35.3%) and the National Center for Research Resources (21 trials, 30.9%).
When trend analysis was performed for the 5 complete years for which the NIBIB existed (2001–2005), a growth rate of 43.7 articles per year (P = .01) was found, by far the fastest growth rate of any NIH institute. The remaining institutes had sufficient data for 10-year analyses. The NCI had the second largest growth rate (13.5 articles per year, P = .003), followed by the National Center for Research Resources (7.9 articles per year, P < .001) and the NINDS (4.7 articles per year, P = .012) (Table 2, Fig 3).
View larger version (18K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 3: Graph of U.S. radiology articles funded by the six largest NIH sponsors of radiology research from 1996 to 2005. NIBIB had the most rapid growth of all institutes. Growth was statistically significant for all institutes shown except NHLBI. NCRR = National Center for Research Resources, NHLBI = National Heart, Lung, and Blood Institute, NIA = National Institute on Aging.
|
|
|
DISCUSSION
|
|---|
It is well established that NIH funding is strongly correlated with radiology department publication output, with NIH funding level alone predicting approximately 86% of publications in academic radiology departments (1). However, it was previously unclear whether the NIH funding was truly causative of publications or merely correlated in some other way. During the past decade, the NIH sponsored less than one-fifth of the 30 059 published research articles originating from U.S. radiology departments. Additionally, despite an increase in the level of NIH funding, the overall number of U.S. radiology publications has remained stable. These findings suggest that NIH funding, while strongly predictive of department publication output, was not directly generating the majority of articles. The NIH may have given grants to departments that were already well funded through industrial or internal mechanisms or to departments that were already highly productive regardless of funding level, thus accounting for the strong correlation between departmental funding and publication output but the relatively low numbers of directly funded publications.
The NIH's influence was not equally distributed among various research methodologies. Expensive and rigorous methodologies, like clinical trials and multicenter studies, were far more likely to receive NIH support than inexpensive methodologies such as case reports, case series, and meta-analyses. General research articles, the category typical of basic science and clinical research, received NIH support at a frequency comparable to that for clinical trials and multicenter studies. Still, more than 70% of clinical trials and 80% of overall articles were funded through non-NIH mechanisms or received no funding at all. This funding gap may have been filled in part by institutional sources and pharmaceutical, biotechnology, and medical device companies. Although the exact impact of private funding of radiology research cannot be evaluated accurately because of unstandardized and inconsistent reporting of non-NIH funding mechanisms, it may be substantial. In the general biomedical literature, industry invests almost twice as much in research as the NIH (8). Whether private industry is as influential in radiology research as in general biomedical research is uncertain, but it clearly plays an important role.
Over time, however, the NIH has played a steadily increasing role in formally sponsoring radiology research. Annual inflation-adjusted extramural grants to radiology departments have been increasing. While the overall number of annually published U.S. radiology articles has been stable, the proportion sponsored by the NIH has been steadily increasing. A survey (9) of published articles in the journals Radiology and AJR in 1990 showed that only 17% of studies received research support and approximately 10% received federal support, mostly from the NIH. In my study, the percentage of NIH-funded articles rose from 14.5% in 1996 to 24.4% in 2005: a proportional growth rate of about 1% annually. Although the 1990 survey was limited to two journals, as compared with the 2673 journals in my study, the 10% proportion of federally funded articles in 1990 is consistent, within one or two percentage points of the predicted NIH funding level in 1990 extrapolated from this study's data. The results of these studies thus agree and indicate that the proportion of NIH-funded radiology research has been increasing since at least 1990.
The growth in NIH support was not evenly distributed among research methodologies. Research with rigorous methodologies experienced the greatest growth in NIH support. In 1996, only 14.6% of U.S. clinical trials were NIH funded. That figure grew to 54.9% in 2005: a 3.5% annual growth in proportion. The funded proportions of clinical trial subtypes, including controlled, randomized-controlled, multicenter, and phase II trials, also grew at statistically significant and disproportionately large rates, ranging from 2.9% to 5.2% annually. Meanwhile, proportionate funding of case reports and series has been flat. This evidence suggests that much of the growth in NIH funding is focused on research with a high degree of scientific value, specifically clinical trials.
There are important policy implications for these results. Because few U.S. radiology articles are directly funded by the NIH, changes in NIH funding are unlikely to have a large impact on the overall number of publications. Certain categories of articles, however, particularly those with high scientific value like clinical trials, are probably sensitive to shifts in funding levels because many rely on NIH funding. Furthermore, because the proportion of NIH-funded clinical trials has been steadily increasing over time, radiology research may be changing fundamentally. Radiology research has been criticized as being more observational and less driven by clinical evidence than research in other specialties (10). This appears to be changing, but at the expense of increased reliance on federal support for research.
The individual NIH institutes support radiology research at vastly different levels. The biggest sponsors are the NCI; NINDS; National Center for Research Resources; National Heart, Lung, and Blood Institute; National Institute on Aging; and NIBIB. The biggest sponsor by far is the NCI, which has funded more than twice as many articles in the past decade as the number-two institute, NINDS. In addition, NCI support is growing at the second-fastest rate. Diagnostic imaging and image-guided intervention are clearly research priorities of the NCI. The Digital Mammographic Imaging Screening Trial and American College of Radiology Imaging Network (ACRIN) were research initiatives prominently mentioned in the NCI's recent budgetary statement to Congress (11). Furthermore, ACRIN is involved in at least 23 multicenter clinical trials (12). Federal funding of cancer-related imaging research has been, and continues to be, strong. In 2005 the NCI budget exceeded $4.8 billion, by far the largest budget of all NIH institutes. It consumed 16.9% of the entire NIH budget, which is shared among 27 institutes and centers (13).
Although the newest NIH institute, the NIBIB is clearly having a large impact on radiology research. The NIBIB was launched with much enthusiasm in the radiology community on the premise that it would make a substantial impact on imaging and bioengineering research (14–16). From 2001 to 2005, it has grown to become the second-largest NIH sponsor of radiology research, although in 2005 the NIBIB's 171 articles were only half of the 341 sponsored by the NCI. Establishment of the NIBIB was an acknowledgment that imaging research was a top priority of the NIH. Its impact on radiology research output has not been evaluated prior to this study to my knowledge. These results are promising and suggest that the NIBIB is fulfilling its role as a prominent supporter of imaging research.
When interpreting these results, certain limitations should be kept in mind. In particular, this analysis uses data from the National Library of Medicine's PubMed database. Failure of PubMed to accurately record article methodologies or stated grant support could bias the results. Furthermore, the failure of authors to indicate their sources of funding could also lead to bias. Generally though, PubMed is considered to be a comprehensive and accurate source of information. In addition, the algorithms used to select radiology articles are not absolutely accurate. Small numbers of nonradiology articles were likely included, potentially biasing the results. When tested, however, the algorithms proved to be 95.8% accurate, and any bias is thus unlikely to result in a substantial change in the results. Finally, this study was limited to articles from radiology departments. Substantial imaging-related research takes place in nonradiology departments. These articles are not included in this analysis.
In conclusion, while overall numbers of published U.S. radiology articles have been stable during the past decade, the number of those funded by the NIH has increased substantially. The impact of NIH funding has been most pronounced for rigorous research methodologies like clinical trials. In particular, controlled, randomized-controlled, multicenter, and phase II trials have seen large proportional funding growth. As of 2005, more than half of published U.S. radiology clinical trials were NIH funded. These rigorous methodologies are likely very sensitive to cuts in NIH funding. The individual NIH institutes supported radiology research variably. By far the largest supporter was the NCI, followed by the NINDS, which indicates the federal government's support of imaging-related cancer and neurologic disease research as top priorities. Grant support from the recently founded NIBIB grew the most rapidly of all NIH institutes. This new institute has quickly assumed a role as an important sponsor of radiology research. Large and increasing proportions of U.S. radiology clinical trials and cancer and neurologic disease research are directly funded by the NIH and thus may be strongly affected by future NIH budget cuts.
|
ADVANCES IN KNOWLEDGE
|
|---|
- U.S. radiology clinical trials are increasingly National Institutes of Health (NIH) funded, with more than half receiving NIH support recently, which indicates that future clinical trials will likely be sensitive to changes in NIH budgetary policy.
- NIH funding support for cancer and neurologic disease research has been particularly strong and is increasing.
- The National Institute of Biomedical Imaging and Bioengineering, created in 2000 to support imaging and bioengineering research, has become the second-largest NIH sponsor of radiology research and has exhibited the fastest growth in supported research of all NIH institutes.
|
IMPLICATION FOR PATIENT CARE
|
|---|
- By providing new insight into radiology research funding and its implications, this study offers guidance for policymakers and investigators with the goal of optimizing the return from future radiology research and ultimately improving patient care.
|
ACKNOWLEDGMENTS
|
|---|
Kimberly E. Applegate, MD, MS, Michael P. Wilson, MD, PhD, and Edward W. Lee, MD, PhD, are thanked for their constructive comments and suggestions.
|
FOOTNOTES
|
|---|
Abbreviations: NCI = National Cancer Institute • NIBIB = National Institute of Biomedical Imaging and Bioengineering • NIH = National Institutes of Health • NINDS = National Institute of Neurological Disorders and Stroke
Author stated no financial relationship to disclose.
|
References
|
|---|
- Itagaki MW, Pile-Spellman J. Factors associated with academic radiology research productivity. Radiology 2005;237:774–780.[Abstract/Free Full Text]
- Giles J, Wadman M. Grants fall victim to NIH success. Nature 2006;443:894–895.[CrossRef][Medline]
- Couzin J. 2008 budget: senators offer sympathetic ear to complaints on NIH's fiscal slide. Science 2007;315:1646.[Abstract/Free Full Text]
- Stat bite: NCI R01 success rates, 1970–2005. J Natl Cancer Inst 2006;98:1437.[Free Full Text]
- Award trends. National Institutes of Health, Office of Extramural Research Web site. http://grants.nih.gov/grants/award/awardtr.htm. Accessed February 7, 2007.
- Consumer Price Indexes. U.S. Department of Labor, Bureau of Labor Statistics Web site. http://www.bls.gov/cpi/. Accessed February 4, 2007.
- About NIBIB. National Institute of Biomedical Imaging and Bioengineering Web site. http://www.nibib.nih.gov/About/MissionHistory. Accessed July 27, 2006.
- Moses H 3rd, Dorsey ER, Matheson DH, Thier SO. Financial anatomy of biomedical research. JAMA 2005;294:1333–1342.[Abstract/Free Full Text]
- Mussurakis S. Financial support for research in radiology: a survey of original investigations published in the AJR and Radiology. AJR Am J Roentgenol 1994;163:973–979.[Abstract/Free Full Text]
- Blackmore CC. The challenge of clinical radiology research. AJR Am J Roentgenol 2001;176:327–331.[Free Full Text]
- Statement on fiscal year 2007 budget request. National Cancer Institute Web site. http://www.cancer.gov/aboutnci/FY07-budget-request. Accessed February 19, 2007.
- Hillman BJ. Doing well by doing good: the ACRIN(R) Fund for Imaging Innovation. J Am Coll Radiol 2006;3:307–308.[CrossRef][Medline]
- NIH Almanac: appropriations. The National Institutes of Health Web site. http://www.nih.gov/about/almanac/appropriations/index.htm. Accessed March 25, 2007.
- Fee CA, Pettigrew RI. National Institute of Biomedical Imaging and Bioengineering: poised for the future. Radiology 2003;229:636–637.[Free Full Text]
- Dunnick NR. Report of the 2002 Intersociety Commission meeting: Radiology 2002—today's research is tomorrow's practice. AJR Am J Roentgenol 2003;180:925–928.[Free Full Text]
- Hendee WR. Opportunities and challenges for the National Institute of Biomedical Imaging and Bioengineering. Radiology 2003;228:316–318.[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
G. P. Krestin
Evaluating the Quality of Radiology Research: What Are the Rules of the Game?
Radiology,
November 1, 2008;
249(2):
418 - 424.
[Full Text]
[PDF]
|
|
|
TOP
ABSTRACT
INTRODUCTION
