HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2361040456v1 236/1/214    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ondategui-Parra, S. Articles by Ros, P. R. Search for Related Content PubMed PubMed Citation Articles by Ondategui-Parra, S. Articles by Ros, P. R.

Use of Productivity and Financial Indicators for Monitoring Performance in Academic Radiology Departments: U.S. Nationwide Survey¹

¹ From the Radiology Management Group, Department of Radiology (S.O.P., J.G.B., K.H.Z., E.N., I.E.G., P.R.R.), and Department of Hospital Administration (S.O.P.), Brigham and Women's Hospital, Harvard Medical School, One Brigham Circle, 1620 Tremont St, Boston, MA 02115; and Department of Health Care Policy, Harvard Medical School, Boston, Mass (K.H.Z.). Received March 9, 2004; revision requested May 20; revision received October 1; accepted November 4. Address correspondence to S.O.P. e-mail: sondateguiparra{at}partners.org).

	ABSTRACT

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 
PURPOSE: To determine how productivity- and finance-related indicators are used by radiology departments to evaluate departmental performance.

MATERIALS AND METHODS: The study met the criteria to be exempt from institutional review board approval. All subjects were informed of the purpose of the study and that their questionnaire responses would be kept confidential. For the study, a survey was sent to 132 members of the Society of Chairmen of Academic Radiology Departments (SCARD) nationwide. The survey was designed to (a) assess organizational information about hospital and radiology departments, (b) determine the types and mean numbers of productivity and financial indicators used by radiology departments, (c) determine how these indicators are used to influence departmental productivity, and (d) assess the reference-standard goals with which each indicator value was compared. A total of 77 variables were studied. Summary statistics, Spearman rank correlation coefficient, and ² analyses were performed.

RESULTS: The response rate was 42% (55 of 132 surveyed SCARD members). The mean number of productivity indicators used by radiology departments was 4.55 ± 2.56 (standard deviation), while the mean number of financial indicators used was 2.89 ± 1.99. Twenty-two (40%) of the 55 responding departments used productivity indicators to monitor and provide feedback to radiologists, hospital leaders, and technical staff members for improved productivity, but only 11 (20%) departments used these indicators to compare personnel performances against specific productivity standards. The most frequent goal (of seven [13%] responding departments) of using the indicators was to increase the examination volume from the previous year by 5%–10%.

CONCLUSION: Academic radiology departments across the United States do not use a standardized set of productivity and financial indicators to measure departmental performance. Examination volume is the most frequently used productivity indicator, whereas general expenses are commonly used as indicators of financial status.

© RSNA, 2005

	INTRODUCTION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 
Today the field of radiology is faced with the challenges of increasing service use, declining reimbursements, and staff shortages (1,2). In such an economically challenging environment, it becomes crucial for radiology departments to monitor their performance so that they can provide a high quality of services while staying within operational boundaries. However, health care is still a major industry in which no unanimously accepted indicators are used to define and measure the quality of services (3). By default, the traditional approach has been to use easily measurable units, such as those related to finance, as indicators for evaluating departmental performance. Today's competitive marketplace, however, demands a more customer-oriented approach to measuring performance (4) and improving financial management (5). It has been suggested that in addition to evaluating financial outcomes, health care managers should assess the intangible assets that affect an organization's basic operations, such as clinical processes, staff skills, and patient satisfaction (6).

The balanced-scorecard approach to monitoring performance that is used in businesses can also be used in radiology (7). Among the indicators used in the balanced-scorecard approach, those that are used to measure financial status and productivity are easily quantifiable and directly related to the activities of the department. Together, these indicators can help to identify strategies for improving performance and increasing revenues. Despite these known advantages, to our knowledge, no established set of indicators to measure productivity or financial status is widely used or has been adopted nationwide by radiology departments.

The recently published results of a U.S. nationwide survey showed that the indicators used most frequently by radiology departments to monitor performance are those related to productivity (8). In addition, productivity indicators and financial indicators were found to be significantly correlated. However, there was not enough detailed information on the use of these indicators provided. Thus, the purposes of the current study were to evaluate the use of productivity and financial indicators in academic radiology departments and to determine the influence of these indicators on departmental performance.

	MATERIALS AND METHODS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 
A multiinstitutional study survey of academic radiology departments across the United States was conducted from May 2002 to November 2002 (8). The study met the criteria to be exempt from approval by the institutional review board of Brigham and Women's Hospital. All subjects who were sent the questionnaire were informed of the purpose of the study and that their responses would be kept confidential. To maintain confidentiality, all responding hospitals were assigned an identification number by the webmaster of Brigham and Women's Hospital, and the study investigators were allowed to access only the final database by using these identification numbers. All of the authors reviewed the questionnaires. Details of the survey design have been described previously (8). Some of the terms commonly used in this article are defined in the Appendix.

General Organization
The survey questionnaire consisted of questions about the basic organizational characteristics of the hospital and the radiology department. These characteristics included (a) the regional location of the facility, (b) the number of beds, (c) the number of examinations performed in the radiology department, and (d) the number of full-time–equivalent (FTE) employees in radiology. The number of beds and number of radiologic examinations performed were used to indicate the sizes of the hospital and the radiology department, respectively.

Productivity Indicators
Nine productivity indicators were studied: examination volume, examination volume per modality, technical relative value units (RVUs), professional RVUs, technical RVUs per FTE employee, professional RVUs per FTE employee, gross charges by modality, collections by FTE employee, and examination volume by resource or device (ie, magnetic resonance imaging or computed tomography unit). The questions pertained to the type of indicators used by the department, the frequency with which the indicators were monitored, and whether the indicators were benchmarked and with whom. Information was also obtained to determine (a) the purposes of monitoring the indicators, (b) the performance standards to which productivity levels were compared, (c) which indicators were considered to be the most useful, (d) how much time was required to obtain information regarding the indicators, and (e) whether the indicator information acquired was communicated to other departmental staff members.

Financial Indicators
The following financial indicators were studied: (a) costs per RVU, (b) average number of RVUs per examination, (c) number of hours worked per RVU, (d) supply costs per RVU, (e) number of charges spent in accounts receivable, (f) collections by modality, and (g) expenses. The questions were intended to determine which of these indicators were used by the radiology departments and how often they were monitored.

Statistical Analyses
Summary statistics, Spearman rank correlation coefficient, and ² analyses were performed. A total of 77 variables that were related to the current investigation were included in the questionnaire. A descriptive analysis was conducted to assess the general organizational characteristics of the institutions surveyed, as well as the types and numbers of productivity and financial indicators most frequently used by the radiology departments. This analysis was also used to evaluate the quality standards used most frequently.

Spearman correlation coefficients (r values) and ² analyses were used to determine whether there were any associations between the numbers and types of productivity and financial indicators used and the following variables: U.S. location of the hospital, hospital size, and radiology department size. Variables were considered to be statistically significant when the P value was less than .05. SPSS version 11.0 (SPSS, Chicago, Ill) was used to perform the statistical analyses.

	RESULTS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 
Throughout this article, the 55 respondents are referred to as hospitals, radiology departments, or members of the Society of Chairmen of Academic Radiology Departments (SCARD). The overall response rate was 42% (55 of 132 surveyed SCARD members). Most of the respondent hospitals were located in the Northeast (20 [36%] of 55) and Midwest (18 [33%] of 55). Additionally, eight (14%) of the 55 hospitals were located in a Southern region; five (9%), in a Southwest region; and four (7%), in a Pacific region. Thirty-two (58%) of the responding hospitals had more than 500 beds in operation, while 21 (38%) had between 200 and 500 beds in operation. Two (4%) responding hospitals had fewer than 200 beds in operation.

The staff of 28 (51%) of the 55 responding radiology departments performed between 200 000 and 400 000 examinations annually, while the staff of 14 (25%) radiology departments performed between 100 000 and 199 999 examinations annually. Of the 55 responding departments, 10 (18%) performed more than 400 000 examinations each year, while three (5%) performed fewer than 100 000 examinations each year.

Number of Indicators
The numbers of productivity indicators used by the departments ranged from zero to nine (mean, 4.55 ± 2.56 [standard deviation]). To monitor financial status, the departments used between zero and seven financial indicators (mean, 2.89 ± 1.99).

The numbers of productivity and financial indicators used were moderately correlated (r = 0.51, P < .001). Furthermore, the number of productivity indicators used correlated minimally with the number of radiologic examinations performed (r = 0.19, P > .05), the number of FTE employees (r = 0.14, P > .05), and the number of hospital beds in operation (r = 0.15, P > .05).

Productivity Indicators
Distribution of indicators.—Among the productivity indicators reportedly used, examination volume and examination volume per modality each were used by 43 (78%) of the 55 responding departments. Professional RVUs and professional RVUs per FTE employee were used by 32 (58%) and 31 (56%) departments, respectively. Indicators such as gross charges per modality and collections by FTE employee were used by 29 (53%) and 27 (49%) departments, respectively. Other indicators such as technical RVUs, technical RVUs per FTE employee, and examination volume by resource or device were used by 16 (29%), 12 (22%), and eight (14%) departments, respectively. Three (5%) of the 55 responding departments did not use any productivity indicator (Table 1).

Eleven (20%) of the 55 responding departments used the indicators for monitoring and providing feedback to radiologists, hospital leaders, and technical staff members to improve productivity and applied standards of reference with which to compare the indicator values. Twenty-two (40%) responding departments used indicators to monitor, provide feedback, and improve productivity but did not apply standards of reference for comparison, while 10 (18%) departments used indicators for monitoring purposes only, without providing feedback or comparing the indicator values with any reference-standard goals.

In terms of examination volume, as well as technical and professional RVUs, the most common goal was to see growth in these indicators by 5%–10% from the previous year. Seven (13%) departments used this target for examination volume, while six (11%) used it for technical and professional RVUs (Table 4).

Benchmarking.—Twenty-six (47%) of the 55 departments used data obtained from various sources to benchmark the productivity indicators. These sources included the Medical Group Management Association, the Radiology Business Managers Association, the Association of Administrators in Academic Radiology and Radiation Oncology, Healthcare Benchmarking Systems International, the University Health System Consortium, and the Association of American Medical Colleges.

² Analysis.—Results of ² analysis showed that the type of productivity indicator used did not vary significantly according to hospital location, hospital size, or radiology department size. Those departments that monitored examination volume and examination volume per modality to improve productivity, as well as compared the values of these indicators with specific reference-standard targets, differed significantly from those departments that used these indicators for productivity monitoring only, without comparing them against reference-standard goals (P < .001). This was true of even those departments that monitored technical RVUs (P = .013), professional RVUs (P = .001), technical RVUs per FTE employee (P = .017), and professional RVUs per FTE employee (P = .015).

Whether a department communicated the information regarding the productivity indicators or benchmarked these indicators, and the individual(s) to whom the indicator information was communicated did not vary significantly according to hospital location, hospital size, or radiology department size. However, there was a significant relationship between the numbers of productivity indicators used and both the communication of indicator information and the individual(s) to whom the information was communicated (P < .04). Compared with the departments that used fewer than five productivity indicators, those departments that used five or more indicators to monitor productivity were more likely to communicate the information to all of the following: hospital leaders, other radiologists, and technical staff members.

Financial Indicators
Use of indicators.—The type of financial indicator used by the departments did not vary significantly according to hospital regional location, hospital size, or number of radiologic examinations performed annually.

Distribution of indicators.—The financial indicators used most frequently were general expenses, which were used by 37 (67%) of the 55 responding departments, and number of charges spent in accounts receivable, which were used by 36 (65%) responding departments. The indicator collections by modality was used by 30 (54%) responding departments, while average number of RVUs per examination was monitored by 16 (29%) departments. Fourteen (26%) responding departments used costs per RVU as an indicator, while 10 (18%) and seven (13%) departments, respectively, used number of worked hours per RVU and supply costs per RVU as financial indicators (Table 4).

Frequency of monitoring.—Almost all of the responding departments monitored the financial indicators monthly or quarterly (Figs 1, 2). Only one of the 55 responding departments monitored collections by modality on a biweekly basis.

View larger version (19K): [in this window] [in a new window] [Download PPT slide]   Figure 1. Relative frequencies of financial indicators that are monitored monthly or quarterly. Graph shows numbers of surveyed departments that monitored the various financial indicators monthly or quarterly. A greater number of departments monitored each indicator monthly rather than quarterly.

View larger version (17K): [in this window] [in a new window] [Download PPT slide]   Figure 2. Means and 95% confidence intervals (CI) of time spent monitoring productivity indicators. Graph shows average numbers of hours per week, and corresponding 95% confidence intervals, required by the departments to monitor productivity indicators, presented according to hospital location. Regional locations are as follows: 1 = Pacific, 2 = Southwest, 3 = Midwest, 4 = Northeast, 5 = South.

	DISCUSSION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

The results of our survey show that the most common indicators used to monitor productivity were those related to examination volume and RVUs. Although most departments use resource-based RVUs and examination volume as the indicators to measure radiologist workload (9,10), these indicators have drawbacks. Examination volume is not a uniform unit of work and cannot be used to compare productivity across departments. Thus, the value of this indicator is limited, and, consequently, many radiology facilities currently use the RVU—a measurement designed to indicate a physician's work activity in uniform units—as an indicator to measure productivity (11).

Previous study results have shown that RVUs are generally not comprehensive measures of a radiologist's clinical productivity (9,11–13). These units indicate only that portion of the clinician's workload that is related directly to patient care. RVUs fail to indicate other activities, such as continuing education, research, participation in conferences, and administrative duties (9). Although these activities are not directly measurable in terms of clinical productivity, they constitute an important part of the radiologist's workload and thus need to be incorporated into the evaluation of radiology staff performance.

Our study results show that there was no comprehensive set of indicators reflecting all aspects of the radiologist's workload that was used unanimously by all departments. There also was no consistency in the frequencies with which these indicators were monitored. Among the indicators used most commonly to monitor productivity, those pertaining to examination volume were monitored either weekly or monthly, while those related to professional RVUs were monitored quarterly.

Another finding of our study was that not all departments communicated information regarding the indicators to all stakeholders. Furthermore, fewer than half the departments used various products for benchmarking. These findings have important implications in terms of increasing productivity and improving performance. Benchmarking is a means of controlling costs by comparing levels of performance with those of "better performers" in the field and identifying opportunities for improvement (14–16). This information obviously needs to be relayed to departmental administrators to help them choose the changes in the work flow processes that are necessary to enhance productivity. The inadequate communication of this information, as evidenced in our study, suggests that the decision-making processes in these departments may be based on insufficient data and may not involve all concerned stakeholders.

Some potential limitations of our study were as follows: (a) the potential bias in the selection of indicators for the questionnaire, (b) the inclusion of only academic radiology departments in the survey, and (c) the response rate. To our knowledge, there was no unanimously accepted set of indicators used to measure radiology performance published in the literature. However, there may be other performance indicators that are monitored by other radiology departments that were not included in our survey. The study was performed in academic centers, and the performance indicators that are relevant in these settings may be different from those used in private practices. We would have to take this fact into account before extending these study results to nonacademic settings.

In conclusion, our study results suggest that most academic radiology departments in the United States do not use a comprehensive set of indicators to monitor productivity and financial status. The indicators most commonly used by academic radiology departments to monitor productivity are examination volume and RVUs, and those used most commonly to monitor financial status are general expenses. These indicators, although useful, have limited value when they are used alone. Hence, it is necessary for radiology departments to use a set of indicators that sufficiently encompasses all aspects of the radiologist's workload. According to our findings, as the next step, it would be useful to develop a "dashboard" of such indicators that could be used uniformly by all radiology departments. The use of a dashboard of indicators would enable a systematic view of a department's level of functioning—from the point of first contact between the physician and the patient to the patient's discharge. It would also facilitate quicker understanding of trends, problems, and opportunities within the department. Dashboards would enable objective comparisons of departmental performance across the country. Further research to evaluate and recommend the appropriate frequency with which to monitor each of these indicators also is necessary.

Finally, we would like to reemphasize the importance of benchmarking these indicators and communicating this information to departmental decision makers. Following the principles of continuous quality improvement, this feedback is essential to helping administrators make the right decisions to increase productivity and identify opportunities for improvement.

	APPENDIX

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 
Benchmark: comparative standard value against which other indicator values may be compared. This value is calculated by using academic specialty–specific billing data to determine statistical comparisons.

Cross-sectional survey: research design in which individuals or events are compared in terms of one or more variables at the same time.

FTE: percentage of full time a provider spends in billable clinical activity.

Management performance indicators: indicators used for management of departmental activities to improve performance.

RVU: nonmonetary unit of measure used to express the time, complexity, and cost of performing a given service relative to other procedures.

	ACKNOWLEDGMENTS

 
The authors thank the following persons for help in conducting the study and preparing this manuscript: Steven E. Seltzer, MD, Lisa A. Intriere, MD, MBA, and Adheet Gogate, MBBS, MPH.

	FOOTNOTES

 

Abbreviations: FTE = full-time equivalent • RVU = relative value unit • SCARD = Society of Chairmen of Academic Radiology Departments

Authors stated no financial relationship to disclose.

Author contributions: Guarantors of integrity of entire study, S.O.P., P.R.R.; study concepts, S.O.P., P.R.R.; study design, S.O.P., P.R.R., K.H.Z.; literature research, J.G.B., S.O.P., E.N., I.E.G.; data acquisition, S.O.P., P.R.R.; data analysis/interpretation, J.G.B., S.O.P., K.H.Z., E.N., I.E.G.; statistical analysis, J.G.B., K.H.Z.; manuscript preparation and editing, J.G.B., S.O.P., K.H.Z., E.N., I.E.G.; manuscript definition of intellectual content, S.O.P., P.R.R.; manuscript revision/review and final version approval, all authors

	References

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION APPENDIX References

 

1. Hillman BJ, Neiman HL. Radiology 2012: radiology and radiologists a decade hence—a strategic analysis for radiology from the second annual American College of Radiology FORUM. Radiology 2003; 227:9–14.[Abstract/Free Full Text]
2. Dummet H. Rising healthcare costs in the US: where will it end? World Markets Research Center, 2001. Available at: http://www.worldmarketsanalysis.com. Accessed May 10, 2003.
3. Mohlenbrock WC. Value-based health care. II. The physician imperative: define, measure and improve health care quality. Physician Exec 1998; 24:47–54.[Medline]
4. Clarke RW, Sucher TO. Benchmarking for the competitive marketplace. J Ambul Care Manage 1999; 22:72–78.[Medline]
5. Hogan C, Sunshine JH. Financial ratios in diagnostic radiology practices: variability and trends. Radiology 2004; 230:774–782.[Abstract/Free Full Text]
6. Oliveira J. The balanced scorecard: an integrative approach to performance evaluation. Healthc Financ Manage 2001; 55:42–46.
7. Zelman WN, Blazer D, Gower M, Bumgarner PO, Cancilla LM. Issues for academic health centers to consider before implementing a balanced-scorecard effort. Acad Med 1999; 74:1269–1277.[Medline]
8. Ondategui-Parra S, Bhagwat JG, Zou KH, et al. Practice management performance indicators in academic radiology departments. Radiology 2004; 233:716–722.[Abstract/Free Full Text]
9. Conoley PM. Productivity of radiologists in 1997: estimates based on analysis of resource-based relative value units. AJR Am J Roentgenol 2000; 175:591–595.[Abstract/Free Full Text]
10. Hsiao WC, Braun P, Becker ER, et al. A national study of resource-based relative value scales for physician services: final report to the Health Care Financing Administration (publication 17-C-98795/1-03). Cambridge, Mass: Harvard School of Public Health, 1998.
11. Sunshine JH, Burkhardt JH. Radiology groups' workload in relative value units and factors affecting it. Radiology 2000; 214:815–822.[Abstract/Free Full Text]
12. Cohen MD, Gunderman RB, Alexander SG. Should pay in radiology be tied to productivity? the case against. Acad Radiol 2002; 9:964–970.[CrossRef][Medline]
13. Arenson RL, Lu Y, Elliott SC, Jovais C, Avrin DE. Measuring the academic radiologist's clinical productivity: survey results for subspecialty sections. Acad Radiol 2001; 8:524–532.[CrossRef][Medline]
14. Devan VR, Williams M. Measuring up: benchmarking tools can enhance executive performance. Trustee 1999; 52:6–9.
15. Zimmerman D. Benchmarking: measuring yourself against the best. Trustee 1999; 52:22.
16. Paz HL, Livingston J. Using a benchmarking system to improve patient care and assist in technology assessment. Physician Exec 1996; 22:10–12.

This article has been cited by other articles:

				A. S. Brody Computed Tomography Scanning in Cystic Fibrosis Research Trials: Practical Lessons from Three Clinical Trials in the United States Proceedings of the ATS, August 1, 2007; 4(4): 350 - 354. [Abstract] [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2361040456v1 236/1/214    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ondategui-Parra, S. Articles by Ros, P. R. Search for Related Content PubMed PubMed Citation Articles by Ondategui-Parra, S. Articles by Ros, P. R.