Statistics For Fun
How do we interpret the results of a null hypothesis significance test?
A researcher tested the null hypothesis that two population means are equal (Ho: mean1 = mean2). A t-test produced p=.010. Assuming that all assumptions of the test have been satisfied, which of the following statements are true and which are false? Why?
- There is a 1% chance of getting a result even more extreme than the observed one when Ho is true.
- There is a 1% likelihood that the result happened by chance.
- There is a 1% chance that the null hypothesis is true.
- There is a 1% chance that the decision to reject Ho is wrong.
- There is a 99% chance that the alternative hypothesis is true, given the observed data.
- A small p value indicates a large effect.
- Rejection of Ho confirms the alternative hypothesis.
- Failure to reject Ho means that the two population means are probably equal.
- Rejecting Ho confirms the quality of the research design.
- If Ho is not rejected, the study is a failure.
- If Ho is rejected in Study 1 but not rejected in Study 2, there must be a moderator variable that accounts for the difference between the two studies.
- There is a 99% chance that a replication study will produce significant results.
- Assuming Ho is true and the study is repeated many times, 1% of these results will be even more inconsistent with Ho than the observed result.
Adapted from Kline, R. B. (2004). Beyond significance testing. Washington, DC: American Psychological Association (pp. 63-69). Dale Berger, CGU 9/04