Abstract
Significance testing is widely used and often criticized. The Task Force on Statistical Inference of the American Psychological Association (TFSI, APA; Wilkinson & TFSI, 1999) addressed the use of significance testing and made recommendations that were incorporated in the fifth edition of the APAPublication Manual (APA, 2001). They emphasized the interpretation of significance testing and the importance of reporting confidence intervals and effect sizes. We examined whether 286Psychonomic Bulletin & Review articles submitted before and after the publication of the TFSI recommendations by APA complied with these recommendations. Interpretation errors when using significance testing were still made frequently, and the new prescriptions were not yet followed on a large scale. Changing the practice of reporting statistics seems doomed to be a slow process.
Article PDF
Similar content being viewed by others
References
American Psychological Association (2001).Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
Bakan, D. (1966). The test of significance in psychological research.Psychological Bulletin,66, 423–437.
Batanero, C. (2000). Controversies around the role of statistical tests in experimental research.Mathematical Thinking & Learning,2, 75–97.
Cohen, J. (1994). The earth is round (p <.05).American Psychologist,49, 997–1003.
Cortina, J. M., &Dunlap, W. P. (1997). On the logic and purpose of significance testing.Psychological Methods,2, 161–172.
Falk, R., &Greenbaum, C. W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception.Theory & Psychology,5, 75–98.
Finch, S., Cumming, G., &Thomason, N. (2001). Reporting of statistical inference in theJournal of Applied Psychology: Little evidence of reform.Educational & Psychological Measurement,61, 181–210.
Finch, S., Thomason, N., &Cumming, G. (2002). Past and future American Psychological Association guidelines for statistical practice.Theory & Psychology,12, 825–853.
Gigerenzer, G. (2004). Mindless statistics.Journal of Socio-Economics,33, 587–606.
Harlow, L. L., Mulaik, S. A., &Steiger, J. H. (1997).What if there were no significance tests? Mahwah, NJ: Erlbaum.
Lecoutre, M.-P., Poitevineau, J., &Lecoutre, B. (2003). Even statisticians are not immune to misinterpretations of null hypothesis tests.International Journal of Psychology,38, 37–45.
Masson, M. E. J., &Loftus, G. R. (2003). Using confidence intervals for graphically based data interpretation.Canadian Journal of Experimental Psychology,57, 203.
Moore, D. S., &McCabe, G. P. (2003).Introduction to the practice of statistics. New York: Freeman.
Mulaik, S. A., Raju, N. S., &Harshman, R. (1997). There is a time and place for significance testing. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.),What if there were no significance tests? (pp. 65–116). Mahwah, NJ: Erlbaum.
Oakes, M. (1986).Statistical inference: A commentary for the social and behavioural sciences. Chichester, U.K.: Wiley.
Rosenthal, R., &Gaito, J. (1963). The interpretation of levels of significance by psychological researchers.Journal of Psychology: Interdisciplinary & Applied,55, 33–38.
Rosnow, R. L., &Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science.American Psychologist,44, 1276–1284.
Rossi, J. S. (1997). A case study in the failure of psychology as a cumulative science: The spontaneous recovery of verbal learning. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.),What if there were no significance tests? (pp. 175–197). Mahwah, NJ: Erlbaum.
Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test.Psychological Bulletin,57, 416–428.
Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers.Psychological Methods,1, 115–129.
Tryon, W. W. (2001). Evaluating statistical difference, equivalence, and indeterminancy using inferential confidence intervals: An integrated alternative method of conducting null hypothesis statistical tests.Psychological Methods,6, 371–386.
Vacha-Haase, T. (2001). Statistical significance should not be considered one of life’s guarantees: Effect sizes are needed.Educational & Psychological Measurement,61, 219–224.
Weisburd, D., Lum, C. M., &Yang, S.-M. (2003). When can we conclude that treatments or programs “don’t work”?Annals of the American Academy of Political & Social Science,587, 31–48.
Wilkinson, L., & the APA Task Force on Statistical Inference (1999). Statistical methods in psychology journals: Guidelines and explanations.American Psychologist,54, 594–604.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hoekstra, R., Finch, S., Kiers, H.A.L. et al. Probability as certainty: Dichotomous thinking and the misuse ofp values. Psychon Bull Rev 13, 1033–1037 (2006). https://doi.org/10.3758/BF03213921
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03213921