HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2263011500v1 226/3/609    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Zou, K. H. Articles by Tempany, C. M. C. Search for Related Content PubMed PubMed Citation Articles by Zou, K. H. Articles by Tempany, C. M. C.

Hypothesis Testing I: Proportions¹

¹ From the Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Mass (K.H.Z., J.R.F., S.G.S., C.M.C.T.); and Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115 (K.H.Z.). Received September 10, 2001; revision requested November 8; revision received December 12; accepted December 19. Supported in part by Public Health Service Grant NIH-U01 CA9398-03 awarded by the National Cancer Institute, Department of Health and Human Services. Address correspondence to K.H.Z. (e-mail: zou@bwh.harvard.edu).

	ABSTRACT

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
Statistical inference involves two analysis methods: estimation and hypothesis testing, the latter of which is the subject of this article. Specifically, Z tests of proportion are highlighted and illustrated with imaging data from two previously published clinical studies. First, to evaluate the relationship between nonenhanced computed tomographic (CT) findings and clinical outcome, the authors demonstrate the use of the one-sample Z test in a retrospective study performed with patients who had ureteral calculi. Second, the authors use the two-sample Z test to differentiate between primary and metastatic ovarian neoplasms in the diagnosis and staging of ovarian cancer. These data are based on a subset of cases from a multiinstitutional ovarian cancer trial conducted by the Radiologic Diagnostic Oncology Group, in which the roles of CT, magnetic resonance imaging, and ultrasonography (US) were evaluated. The statistical formulas used for these analyses are explained and demonstrated. These methods may enable systematic analysis of proportions and may be applied to many other radiologic investigations.

© RSNA, 2003

Index terms: Statistical analysis

	INTRODUCTION

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
Statistics often involve a comparison of two values when one or both values are associated with some uncertainty. The purpose of statistical inference is to aid the clinician, researcher, or administrator in reaching a conclusion concerning a population by examining a sample from that population. Statistical inference consists of two components, estimation and hypothesis testing, and the latter component is the main focus of this article.

Estimation can be carried out on the basis of sample values from a larger population (1). Point estimation involves the use of summary statistics, including the sample mean and SD. These values can be used to estimate intervals, such as the 95% confidence level. For example, by using summary statistics, one can determine the sensitivity or specificity of the size and location of a ureteral stone for prediction of the clinical management required. In a study performed by Fielding et al (2), it was concluded that stones larger than 5 mm in the upper one-third of the ureter were very unlikely to pass spontaneously.

In contrast, hypothesis testing enables one to quantify the degree of uncertainty in sampling variation, which may account for the results that deviate from the hypothesized values in a particular study (3,4). For example, hypothesis testing would be necessary to determine if ovarian cancer is more prevalent in nulliparous women than in multiparous women.

It is important to distinguish between a research hypothesis and a statistical hypothesis. The research hypothesis is a general idea about the nature of the clinical question in the population of interest. The primary purpose of the statistical hypothesis is to establish the basis for tests of significance. Consequently, there is also a difference between a clinical conclusion based on a clinical hypothesis and a statistical conclusion of significance based on a statistical hypothesis. In this article, we will focus on statistical hypothesis testing only.

In this article we review and demonstrate the hypothesis tests for both a single proportion and a comparison of two independent proportions. The topics covered may provide a basic understanding of the quantitative approaches for analyzing radiologic data. Detailed information on these concepts may be found in both introductory (5,6) and advanced textbooks (7–9). Related links on the World Wide Web are listed in Appendix A.

	STATISTICAL HYPOTHESIS TESTING BASICS

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
A general procedure is that of calculating the probability of observing the difference between two values if they really are not different. This probability is called the P value, and this condition is called the null hypothesis (H₀). On the basis of the P value and whether it is low enough, one can conclude that H₀ is not true and that there really is a difference. This act of conclusion is in some ways a "leap of faith," which is why it is known as statistical significance. In the following text, we elaborate on these key concepts and the definitions needed to understand the process of hypothesis testing.

There are five steps necessary for conducting a statistical hypothesis test: (a) formulate the null (H₀) and alternative (H₁) hypotheses, (b) compute the test statistic for the given conditions, (c) calculate the resulting P value, (d) either reject or do not reject H₀ (reject H₀ if the P value is less than or equal to a prespecified significance level [typically .05]; do not reject H₀ if the P value is greater than this significance level), and (e) interpret the results according to the clinical hypothesis relevant to H₀ and H₁. Each of these steps are discussed in the following text.

Null and Alternative Hypotheses
In general, H₀ assumes that there is no association between the predictor and outcome variables in the study population. In such a case, a predictor (ie, explanatory or independent) variable is manipulated, and this may have an effect on another outcome or dependent variable. For example, to determine the effect of smoking on blood pressure, one could compare the blood pressure levels in nonsmokers, light smokers, and heavy smokers.

It is mathematically easier to frame hypotheses in null and alternative forms, with H₀ being the basis for any statistical significance test. Given the H₀ of no association between a predictor variable and an outcome variable, a statistical hypothesis test can be performed to estimate the probability of an association due to chance that is derived from the available data. Thus, one never accepts H₀, but rather one rejects it with a certain level of significance.

In contrast, H₁ makes a claim that there is an association between the predictor and outcome variables. One does not directly test H₁, which is by default accepted when H₀ is rejected on the basis of the statistical significance test results.

One- and Two-sided Tests
The investigator must also decide whether a one- or two-sided test is most suitable for the clinical question (4). A one-sided H₁ test establishes the direction of the association between the predictor and the outcome—for example, that the prevalence of ovarian cancer is higher in nulliparous women than in parous women. In this example, the predictor is parity and the outcome is ovarian cancer. However, a two-sided H₁ test establishes only that an association exists without specifying the direction—for example, the prevalence of ovarian cancer in nulliparous women is different (ie, either higher or lower) from that in parous women. In general, most hypothesis tests involve two-sided analyses.

Test Statistic
The test statistic is a function of summary statistics computed from the data. A general formula for many such test statistics is as follows: test statistic = (relevant statistic - hypothesized parameter value)/(standard error of the relevant statistic), where the relevant statistics and standard error are calculated on the basis of the sample data. The standard error is the indicator of variability, and much of the complexity of the hypothesis test involves estimating the standard error correctly. H₀ is rejected if the test statistic exceeds a certain level (ie, critical value).

For example, for continuous data, the Student t test is most often used to determine the statistical significance of an observed difference between mean values with unknown variances. On the basis of large samples with underlying normal distributions and known variances (5), the Z test of two population means is often conducted. Similar to the t test, the Z test involves the use of a numerator to compare the difference between the sample means of the two study groups with the difference that would be expected with H₀, that is, zero difference. The denominator includes the sample size, as well as the variances, of each study group (5).

Once the Z value is calculated, it can be converted into a probability statistic by means of locating the P value in a standard reference table. The Figure illustrates a standard normal distribution (mean of 0, variance of 1) of a test statistic, Z, with two rejection regions that are either below -1.96 or above 1.96. Two hypothetical test statistic values, -0.5 and 2.5, which lie outside and inside the rejection regions, respectively, are also included. Consequently, one does not reject H₀ when Z equals -0.5, but one does reject H₀ when Z equals 2.5.

View larger version (18K): [in this window] [in a new window] [Download PPT slide]   Graph illustrates the normal distribution of the test statistic Z in a two-sided hypothesis test. Under H0, Z has a standard normal distribution, with a mean of 0 and a variance of 1. The critical values are fixed at ±1.96, which corresponds to a 5% significance level (ie, type I error) under H0. The rejection regions are the areas marked with oblique lines under the two tails of the curve, and they correspond to any test statistic lying either below -1.96 or above +1.96. Two hypothetical test statistic values, -0.5 and 2.5, result in not rejecting or rejecting H0, respectively.

Type I and II Errors
Two types of errors can occur in hypothesis testing: A type I error (significance level ) represents the probability that H₀ was erroneously rejected when in fact it is true in the underlying population. Note that the P value is not the same as the value, which represents the significance level in a type I error. The significance level is prespecified (5% conventionally), whereas the P value is computed on the basis of the data and thus reflects the strength of the rejection of H₀ on the test statistic. A type II error (significance level ß) represents the probability that H₀ was erroneously retained when in fact H₁ is true in the underlying population. There is always a trade-off between these two types of errors, and such a relationship is similar to that between sensitivity and specificity in the diagnostic literature (Table) (10). The probability 1 - ß is the statistical power and is analogous to the sensitivity of a diagnostic test, whereas the probability 1 - is analogous to the specificity of a diagnostic test.

	STATISTICAL TESTS OF PROPORTIONS: THE Z TEST

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

When sample sizes are large, the approximate normality assumptions hold for both the sample proportion and the test statistic. In the test of a single proportion () based on a sample of n independent trials at a hypothesized success probability of ₀ (the hypothesized proportion), both n₀ and n(1 - ₀) need to be at least 5 (Appendix B). In the comparison of two proportions, ₁ and ₂, based on two independent sample sizes of n₁ and n₂ independent trials, respectively, both n₁ and n₂ need to be at least 30 (Appendix C) (5). The test statistic is labeled Z, and, hence, the analysis is referred to as the Z test of a proportion. Other exact hypothesis-testing methods are available if these minimum numbers are not met.

Furthermore, the Z and Student t tests both are parametric hypothesis tests—that is, they are based on data with an underlying normal distribution. There are many situations in radiology research in which the assumptions needed to use a parametric test do not hold. Therefore, nonparametric tests must be considered (9). These statistical tests will be discussed in a future article.

	TWO RADIOLOGIC EXAMPLES

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
One-Sample Z Test of a Single Proportion
Fielding et al (2) evaluated the unenhanced helical CT features of 100 ureteral calculi, 71 of which passed spontaneously and 29 of which required intervention. According to data in the available literature (11–13), approximately 80% of the stones smaller than 6 mm in diameter should have passed spontaneously. Analysis of the data in the Fielding et al study revealed that of 66 stones smaller than 6 mm, 57 (86%) passed spontaneously. To test if the current finding agrees with that in the literature, we conduct a statistical hypothesis test with five steps:

1. H₀ is as follows: 80% of the ureteral stones smaller than 6 mm will pass spontaneously ( = 0.80). H₁ is as follows: The proportion of the stones smaller than 6 mm that pass spontaneously does not equal 80%—that is, it is either less than or greater than 80% ( 0.80). This is therefore a two-sided hypothesis test.

2. The test statistic Z is calculated to be 1.29 on the basis of the results of the Z test of a single proportion (5).

3. The P value, .20, is the sum of the two tail probabilities of a standard normal distribution for which the Z values are beyond ±1.29 (Figure).

4. Because the P value, .20, is greater than the significance level of 5%, H₀ is not rejected.

5. Therefore, our data support the belief that 80% of the stones smaller than 6 mm in diameter will pass spontaneously, as reported in the literature. Thus, H₀ is not rejected, given the data at hand. Consequently, it is possible that a type II error will occur if the true proportion in the population does not equal 80%.

Two-Sample Z Test to Compare Two Independent Proportions
Brown et al (14) hypothesized that the imaging appearances (eg, multilocularity) of primary ovarian tumors and metastatic tumors to the ovary might be different. Data were obtained from 280 patients who had an ovarian mass and underwent US in the Radiologic Diagnostic Oncology Group (RDOG) ovarian cancer staging trial (15,16). The study results showed that 30 (37%) of 81 primary ovarian cancers, as compared with three (13%) of 24 metastatic neoplasms, were multilocular at US. To test if the respective underlying proportions are different, we conduct a statistical hypothesis test with five steps:

1. H₀ is as follows: There is no difference between the proportions of multilocular metastatic tumors (₁) and multilocular primary ovarian tumors (₂) among the primary and secondary ovarian cancers—that is, ₁ - ₂ = 0. H₁ is as follows: There is a difference in these proportions: One is either less than or greater than the other—that is, ₁ - ₂ 0. Thus, a two-sided hypothesis test is conducted.

2. The test statistic Z is calculated to be 2.27 on the basis of the results of the Z test to compare two independent proportions (5).

3. The P value, .02, is the sum of the two tail probabilities of a standard normal distribution for which the Z values are beyond ±2.27 (Figure).

4. Because the P value, .02, is less than the significance level of 5%, H₀ is rejected.

5. Therefore, there is a statistically significant difference between the proportion of multilocular masses in patients with primary tumors and that in patients with metastatic tumors.

	SUMMARY AND REMARKS

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
In this article, we reviewed the hypothesis tests of a single proportion and for comparison of two independent proportions and illustrated the two test methods by using data from two prospective clinical trials. Formulas and program codes are provided in the Appendices. With large samples, the normality of a sample proportion and test statistic can be conveniently assumed when conducting Z tests (5). These methods are the basis for much of the scientific research conducted today; they allow us to make conclusions about the strength of research evidence, as expressed in the form of a probability.

Alternative exact hypothesis-testing methods are available if the sample sizes are not sufficiently large. In the case of a single proportion, the exact binomial test can be conducted. In the case of two independent proportions, the proposed large-sample Z test is equivalent to a test based on contingency table (ie, ²) analysis. When large samples are not available, however, the Fisher exact test based on contingency table analysis can be adopted (8,17–19). For instance, in the clinical example involving data from the RDOG study, the sample of 24 metastatic neoplasms is slightly smaller than the required sample of 30 neoplasms, and, thus, use of the exact Fisher test may be preferred.

The basic concepts and methods reviewed in this article may be applied to similar inferential and clinical trial design problems related to counts and proportions. More complicated statistical methods and study designs may be considered, but these are beyond the scope of this tutorial article (20–24). A list of available software packages can be found by accessing the Web links given in Appendix A.

	APPENDIX A

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
Statistical Resources Available on the World Wide Web
The following are links to electronic textbooks on statistics: www.davidmlane.com /hyperstat/index.html, www.statsoft.com/textbook /stathome.html, www.ruf.rice.edu/lane/rvls.html, www.bmj.com:/collections/statsbk/index.shtml, and espse.ed.psu.edu/statistics/investigating .htm. In addition, statistical software packages are available at the following address: www.amstat.org/chapters/alaska/resources .htm.

	APPENDIX B

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 
Testing a Single Proportion by Using a One-Sample Z Test
Let be a population proportion to be tested (Table B1). The procedure for deciding whether or not to reject H₀ is as follows: = ₀; this is based on the results of a one-sided, one-sample Z test at the significance level of with n independent trials(Table B1). The observed number of successes is x, and, thus, the sample proportion of successes is p = x/n. In our first clinical example, that in which the unenhanced helical CT features of 100 ureteral calculi were evaluated (2), = 0.80, n = 66, x = 57, and p = 66/57 (0.86).

	APPENDIX C

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

	ACKNOWLEDGMENTS

 
We thank Kimberly E. Applegate, MD, MS, and Philip E. Crewson, PhD, for their constructive comments on earlier versions of this article.

	FOOTNOTES

 
² Current address: Department of Radiology, University of North Carolina at Chapel Hill.

Abbreviations: H₀ = null hypothesis, H₁ = alternative hypothesis, RDOG = Radiologic Diagnostic Oncology Group

	REFERENCES

TOP ABSTRACT INTRODUCTION STATISTICAL HYPOTHESIS TESTING... STATISTICAL TESTS OF... TWO RADIOLOGIC EXAMPLES SUMMARY AND REMARKS APPENDIX A APPENDIX B APPENDIX C REFERENCES

 

1. Altman DG. Statistics in medical journals: some recent trends. Statist Med 2000; 1g:3275-3289.
2. Fielding JR, Silverman SG, Samuel S, Zou KH, Loughlin KR. Unenhanced helical CT of ureteral stones: a replacement for excretory urography in planning treatment. AJR Am J Roentgenol 1998; 171:1051-1053.[Abstract/Free Full Text]
3. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed) 1986; 292:746-750.
4. Bland JM, Altman DF. One and two sided tests of significance. BMJ 1994; 309:248.[Free Full Text]
5. Goldman RN, Weinberg JS. Statistics: an introduction Englewood Cliffs, NJ: Prentice Hall, 1985; 334-353.
6. Hulley SB, Cummings SR. Designing clinical research: an epidemiologic approach Baltimore, Md: Williams & Wilkins, 1988; 128-138, 216–217.
7. Freund JE. Mathematical statistics 5th ed. Englewood Cliffs, NJ: Prentice Hall, 1992; 425-430.
8. Fleiss JL. Statistical methods for rates and proportions 2nd ed. New York, NY: Wiley, 1981; 1-49.
9. Gibbons JD. Sign tests. In: Kotz S, Johnson NL, eds. Encyclopedia of statistical sciences. New York, NY: Wiley, 1982; 471-475.
10. Browner WS, Newman TB. Are all significant p values created equal?The analogy between diagnostic tests and clinical research. JAMA 1987; 257:2459-2463.[Abstract]
11. Drach GW. Urinary lithiasis: etiology, diagnosis and medical management. In: Walsh PC, Staney TA, Vaugham ED, eds. Campbell’s urology. 6th ed. Philadelphia, Pa: Saunders, 1992; 2085-2156.
12. Segura JW, Preminger GM, Assimos DG, et al. Ureteral stones: clinical guidelines panel summary report on the management of ureteral calculi. J Urol 1997; 158:1915-1921.[CrossRef][Medline]
13. Motola JA, Smith AD. Therapeutic options for the management of upper tract calculi. Urol Clin North Am 1990; 17:191-206.[Medline]
14. Brown DL, Zou KH, Tempany CMC, et al. Primary versus secondary ovarian malignancy: imaging findings of adnexal masses in the Radiology Diagnostic Oncology Group study. Radiology 2001; 219:213-218.[Abstract/Free Full Text]
15. Kurtz AB, Tsimikas JV, Tempany CMD, et al. Diagnosis and staging of ovarian cancer: comparative values of Doppler and conventional US, CT, and MR imaging correlated with surgery and histopathologic analysis—report of the Radiology Diagnostic Oncology Group. Radiology 1999; 212:19-27.[Abstract/Free Full Text]
16. Tempany CM, Zou KH, Silverman SG, Brown DL, Kurtz AB, McNeil BJ. Stating of ovarian cancer: comparison of imaging modalities—report from the Radiology Diagnostic Oncology Group. Radiology 2000; 215:761-767.[Abstract/Free Full Text]
17. Agresti A. Categorical data analysis New York, NY: Wiley, 1990; 8-35.
18. Joe H. Extreme probabilities for contingency tables under row and column independence with application to Fisher’s exact test. Comm Stat A Theory Methods 1988; 17:3677-3685.
19. MathSoft/Insightful. S-Plus 4 guide to statistics Seattle, Wash: MathSoft, 1997; 89-96. Available at: http://www.insightful.com/products.splus.
20. Lehmann EL, Casella G. Theory of point estimation New York, NY: Springer Verlag, 1998.
21. Lehmann EL. Testing statistical hypotheses 2nd ed. New York, NY: Springer Verlag, 1986.
22. Hettmansperger TP. Statistical inference based on ranks Malabar, Fla: Krieger, 1991.
23. Joseph L, Du Berger R, Belisle P. Bayesian and mixed Bayesian/likelihood criteria for sample size determination. Stat Med 1997; 16:769-781.[CrossRef][Medline]
24. Zou KH, Norman SL. On determination of sample size in hierarchical binomial models. Stat Med 2001; 20:2163-2182.[CrossRef][Medline]

This article has been cited by other articles:

				V. Bajpai, K. H. Lee, B. Kim, K. J. Kim, T. J. Kim, Y. H. Kim, and H. S. Kang Differences in Compression Artifacts on Thin- and Thick-Section Lung CT Images Am. J. Roentgenol., August 1, 2008; 191(2): W38 - W43. [Abstract] [Full Text] [PDF]

				N. Petrick, M. Haider, R. M. Summers, S. C. Yeshwant, L. Brown, E. M. Iuliano, A. Louie, J. R. Choi, and P. J. Pickhardt CT Colonography with Computer-aided Detection as a Second Reader: Observer Performance Study Radiology, December 1, 2007; 246(1): 148 - 156. [Abstract] [Full Text] [PDF]

				R. S. Stafford, V. Monti, C. D. Furberg, and J. Ma Long-Term and Short-Term Changes in Antihypertensive Prescribing by Office-Based Physicians in the United States Hypertension, August 1, 2006; 48(2): 213 - 218. [Abstract] [Full Text] [PDF]

				M. W. Ragozzino, G. Brancatelli, V. Vilgrain, M. P. Federle, F. Uzan, M. Zappa, and Y. Menu Biases Likely Invalidate the Conclusions [letter] * Dr Brancatelli and colleagues respond: Radiology, June 1, 2004; 231(3): 926 - 927. [Full Text] [PDF]

				R. Tello and P. E. Crewson Hypothesis Testing II: Means Radiology, April 1, 2003; 227(1): 1 - 4. [Abstract] [Full Text] [PDF]

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2263011500v1 226/3/609    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Zou, K. H. Articles by Tempany, C. M. C. Search for Related Content PubMed PubMed Citation Articles by Zou, K. H. Articles by Tempany, C. M. C.