Skip to content


Personal tools
You are here: Home » SSCC » gangc's Home » Statistics For Fun

Statistics For Fun

Document Actions

How do we interpret the results of a null hypothesis significance test?

A researcher tested the null hypothesis that two population means are equal (Ho: mean1 = mean2). A t-test produced p=.010. Assuming that all assumptions of the test have been satisfied, which of the following statements are true and which are false? Why?

  1. There is a 1% chance of getting a result even more extreme than the observed one when Ho is true.
  2. There is a 1% likelihood that the result happened by chance.
  3. There is a 1% chance that the null hypothesis is true.
  4. There is a 1% chance that the decision to reject Ho is wrong.
  5. There is a 99% chance that the alternative hypothesis is true, given the observed data.
  6. A small p value indicates a large effect.
  7. Rejection of Ho confirms the alternative hypothesis.
  8. Failure to reject Ho means that the two population means are probably equal.
  9. Rejecting Ho confirms the quality of the research design.
  10. If Ho is not rejected, the study is a failure.
  11. If Ho is rejected in Study 1 but not rejected in Study 2, there must be a moderator variable that accounts for the difference between the two studies.
  12. There is a 99% chance that a replication study will produce significant results.
  13. Assuming Ho is true and the study is repeated many times, 1% of these results will be even more inconsistent with Ho than the observed result.

Adapted from Kline, R. B. (2004). Beyond significance testing. Washington, DC: American Psychological Association (pp. 63-69). Dale Berger, CGU 9/04

Created by Alex Clark
Last modified 2005-02-14 14:56

Powered by Plone

This site conforms to the following standards: