Skip to main content
SpringerLink
Log in

Retest reliability of the parameters of the Ratcliff diffusion model

  • Original Article
  • Published:
Psychological Research Aims and scope Submit manuscript

Abstract

In the recent years, there is a growing interest to use the Ratcliff Diffusion Model (1978) for diagnostic purposes as the parameters of the model capture interindividual differences in specific cognitive processes. The parameters are estimated using reaction time data from binary classification tasks. For a potential diagnostic application of parameter values sufficient reliability is a necessary precondition. In two studies, each with two sessions separated by 1 week, the retest reliability of the diffusion model parameters was assessed. In Study 1, 105 participants completed a lexical decision task and a recognition memory task. In Study 2, 128 participants worked on an associative priming task. Results show that the reliability of the main parameters of the Ratcliff Diffusion Model (in particular of the speed of information accumulation and the threshold separation with rs > 0.70 for all three tasks) is satisfying. Besides, we analyzed the influence of the number of trials on the retest reliability using different estimation methods (Kolmogorov–Smirnov, Maximum Likelihood, Chi-square and EZ) and both empirical and simulated datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Noteworthy, in unpublished studies from our lab, we failed to find correlations of threshold separation with self-reported impulsivity using standard speeded response time tasks (cf. also Stahl et al., 2014). However, when using more difficult tasks that required a long duration of information accumulation (RT > 5 s) weak to moderate correlations emerged.

  2. Words had frequencies below 5 per million (CELEX; Baayen, Piepenbrock, & Gulikers, 1995) and—at the same time—a frequency class of 14 or 15 (online dictionary project of the university of Leipzig in November 2014, see http://wortschatz.unileipzig.de), indicating that the word “der” (“the”) is used 214 or 215 times as often in German language as our stimuli.

  3. As EZ cannot be applied to datasets with an accuracy rate of 100 %, we applied an edge correction method which has also been used by Wagenmakers et al. (2007): \({\text{accuracy}} = 1 - \frac{1}{2 \times n}\), with n being the number of trials. Similarly, in Study 3 we additionally used a correction for a few datasets due to an observed accuracy of 50 % in one condition (\({\text{accuracy}} = 0.5 + \frac{1}{2 \times n}\)).

  4. Based on the findings by Voss et al. (2013), we did not expect the d-parameter of the diffusion model (Voss et al., 2010) to be influenced by the prime type. Besides, estimation of this parameter requires very high trial numbers (Voss et al., 2010).

  5. Results for non-word targets are also presented in Table 3. Note, that the findings are similar to those reported by Voss et al. (2013).

  6. In each condition, fast-dm requires at least 10 trials (independent of the type of response) for ML and KS estimation and 12 trials (of the same response) for CS estimation (Voss et al., 2015). Thus, for the data of the APT due to the higher number of conditions no retest coefficients could be computed for 32 trials and for CS neither for 48 trials.

  7. Construct-samples is part of fast-dm (Voss et al., 2015). For the simulation of datasets we used a high precision setting of p = 4.

  8. Note that, as already mentioned, it is possible that the contamination in the empiric data is different from the type and amount of contamination that we assumed for the simulation of data. Thus, it could be that the estimation of the drift rate suffers less from contamination than for example the estimation of t 0 and that not (only) the stability of the parameters is responsible for the different distances between the lines of simulated and empiric data. Our study, thus, only allows getting an approximate idea of the state proportions. A clear disentangling of state and trait proportions would require larger samples of participants and data points.

  9. Results are very similar for Session 2.

References

  • Allen, P. A., Lien, M.-C., Ruthruff, E., & Voss, A. (2014). Multitasking and aging: Do older adults benefit from performing a highly practiced task? Experimental Aging Research, 40(3), 280–307.

    Article  PubMed  Google Scholar 

  • Arnold, N. R., Bröder, A., & Bayen, U. J. (2015). Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods. Psychological Research, 79(5), 882–898. doi:10.1007/s00426-014-0608-y.

    Article  PubMed  Google Scholar 

  • Aschenbrenner, A. J., Balota, D. A., Gordon, B. A., Ratcliff, R., & Morris, J. C. (2016). A diffusion model analysis of episodic recognition in preclinical individuals with a family history for Alzheimer’s disease: The adult children study. Neuropsychology, 30(2), 225–238. doi:10.1037/neu0000222.

    Article  PubMed  PubMed Central  Google Scholar 

  • Baayen, R. H., Piepenbrock, R., & Gulikers, L. (1995). The CELEX lexical database (CD-ROM). Linguistic Data Consortium. Philadelphia, PA: University of Pennsylvania.

    Google Scholar 

  • Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459.

    Article  PubMed  Google Scholar 

  • Bock, O., Baetge, I., & Nicklisch, A. (2014). hroot: Hamburg registration and organization online tool. European Economic Review, 71, 117–120.

    Article  Google Scholar 

  • Borkenau, P., & Ostendorf, F. (2008). NEO-Fünf-Faktoren Inventar: nach Costa u. McCrae; NEO-FFI: Hogrefe, Verlag f. Psychologie.

  • Bowen, H. J., Spaniol, J., Patel, R., & Voss, A. (2016). A Diffusion Model Analysis of Decision Biases Affecting Delayed Recognition of Emotional Stimuli. Plos ONE, 11(1), 1–20. doi:10.1371/journal.pone.0146769.

    Article  Google Scholar 

  • Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E.-J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin & Review, 16(6), 1026–1036. doi:10.3758/16.6.1026.

    Article  Google Scholar 

  • Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: the implicit association test. Journal of Personality and Social Psychology, 74(6), 1464.

    Article  PubMed  Google Scholar 

  • Guilford, J. P. (1954). Psychometric methods (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Hintzman, D. L., & Curran, T. (1997). Comparing retrieval dynamics in recognition memory and lexical decision. Journal of Experimental Psychology: General, 126(3), 228–247. doi:10.1037/0096-3445.126.3.228.

    Article  Google Scholar 

  • Klauer, K. C., & Dittrich, K. (2010). From sunshine to double arrows: An evaluation window account of negative compatibility effects. Journal of Experimental Psychology: General, 139(3), 490.

    Article  Google Scholar 

  • Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. University of Florida, Gainesville, FL.

  • Leite, F. P., & Ratcliff, R. (2011). What cognitive processes drive response biases? A diffusion model analysis. Judgment and Decision Making, 6(7), 651–687.

    Google Scholar 

  • Lerche, V., & Voss, A. (2016). Model complexity in diffusion modeling: Benefits of making the model more parsimonious. Manuscript submitted for publication.

  • Lerche, V., Voss, A., & Nagler, M. (2015). How Many Trials are Required for Robust Parameter Estimation in Diffusion Modeling? A Comparison of Different Estimation Algorithms. Manuscript submitted for publication.

  • Metin, B., Roeyers, H., Wiersema, J. R., van der Meere, J. J., Thompson, M., & Sonuga-Barke, E. (2013). ADHD performance reflects inefficient but not impulsive information processing: A diffusion model analysis. Neuropsychology, 27(2), 193–200. doi:10.1037/a0031533.

    Article  PubMed  Google Scholar 

  • Neumann, R., & Strack, F. (2000). Approach and avoidance: the influence of proprioceptive and exteroceptive cues on encoding of affective information. Journal of Personality and Social Psychology, 79(1), 39.

    Article  PubMed  Google Scholar 

  • Pe, M. L., Vandekerckhove, J., & Kuppens, P. (2013). A diffusion model account of the relationship between the emotional flanker task and rumination and depression. Emotion, 13(4), 739–747. doi:10.1037/a0031628.

    Article  PubMed  Google Scholar 

  • Petrov, A. A., Van Horn, N. M., & Ratcliff, R. (2011). Dissociable perceptual-learning mechanisms revealed by diffusion-model analysis. Psychonomic Bulletin & Review, 18(3), 490–497. doi:10.3758/s13423-011-0079-8

    Article  Google Scholar 

  • R Core Team. (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/.

  • Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108. doi:10.1037/0033-295x.85.2.59.

    Article  Google Scholar 

  • Ratcliff, R., & Childers, R. (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision, 2(4), 237–279. doi:10.1037/dec0000030.

    Article  Google Scholar 

  • Ratcliff, R., Gomez, P., & McKoon, G. (2004a). A Diffusion Model Account of the Lexical Decision Task. Psychological Review, 111(1), 159–182. doi:10.1037/0033-295x.111.1.159.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4), 873–922. doi:10.1162/neco.2008.12-06-420.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., & McKoon, G. (2015). Aging effects in item and associative recognition memory for pictures and words. Psychology and Aging, 30(3), 669–674.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9(5), 347–356. doi:10.1111/1467-9280.00067.

    Article  Google Scholar 

  • Ratcliff, R., Spieler, D., & McKoon, G. (2000). Explicitly modeling the effects of aging on response time. Psychonomic Bulletin & Review, 7(1), 1–25.

    Article  Google Scholar 

  • Ratcliff, R., Thapar, A., Gomez, P., & McKoon, G. (2004b). A diffusion model analysis of the effects of aging in the lexical-decision task. Psychology and Aging, 19(2), 278.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., Thapar, A., & McKoon, G. (2001). The effects of aging on reaction time in a signal detection task. Psychology and Aging, 16(2), 323.

    Article  PubMed  Google Scholar 

  • Ratcliff, R., Thapar, A., & McKoon, G. (2004c). A diffusion model analysis of the effects of aging on recognition memory. Journal of Memory and Language, 50(4), 408–424.

    Article  Google Scholar 

  • Ratcliff, R., Thapar, A., & McKoon, G. (2007). Application of the diffusion model to two-choice tasks for adults 75-90 years old. Psychology and Aging, 22(1), 56–66. doi:10.1037/0882-7974.22.1.56.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., Thapar, A., & McKoon, G. (2010). Individual differences, aging, and IQ in two-choice tasks. Cognitive Psychology, 60(3), 127–157. doi:10.1016/j.cogpsych.2009.09.001.

    Article  PubMed  Google Scholar 

  • Ratcliff, R., Thapar, A., & McKoon, G. (2011). Effects of aging and IQ on item and associative memory. Journal of Experimental Psychology: General, 140(3), 464.

    Article  Google Scholar 

  • Ratcliff, R., Thompson, C. A., & McKoon, G. (2015). Modeling individual differences in response time and accuracy in numeracy. Cognition, 137, 115–136. doi:10.1016/j.cognition.2014.12.004.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9(3), 438–481. doi:10.3758/bf03196302.

    Article  Google Scholar 

  • Schmiedek, F., Oberauer, K., Wilhelm, O., Süß, H.-M., & Wittmann, W. W. (2007). Individual differences in components of reaction time distributions and their relations to working memory and intelligence. Journal of Experimental Psychology: General, 136(3), 414–429. doi:10.1037/0096-3445.136.3.414.

    Article  Google Scholar 

  • Schmitz, F., & Voss, A. (2012). Decomposing task-switching costs with the diffusion model. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 222–250. doi:10.1037/a0026003.

    PubMed  Google Scholar 

  • Schmitz, F., & Voss, A. (2014). Components of task switching: A closer look at task switching and cue switching. Acta Psychologica, 151, 184–196. doi:10.1016/j.actpsy.2014.06.009.

    Article  PubMed  Google Scholar 

  • Schubert, A.-L., Hagemann, D., Voss, A., Schankin, A., & Bergmann, K. (2015). Decomposing the relationship between mental speed and mental abilities. Intelligence, 51, 28–46. doi:10.1016/j.intell.2015.05.002.

    Article  Google Scholar 

  • Spaniol, J., Madden, D. J., & Voss, A. (2006). A Diffusion Model Analysis of Adult Age Differences in Episodic and Semantic Long-Term Memory Retrieval. Journal of Experimental Psychology: Learning, Memory & Cognition, 32(1), 101–117. doi:10.1037/0278-7393.32.1.101.

    Article  Google Scholar 

  • Spaniol, J., Voss, A., & Grady, C. L. (2008). Aging and emotional memory: Cognitive mechanisms underlying the positivity effect. Psychology and Aging, 23(4), 859–872. doi:10.1037/a0014218.

    Article  PubMed  Google Scholar 

  • Stahl, C., Voss, A., Schmitz, F., Nuszbaum, M., Tüscher, O., Lieb, K., & Klauer, K. C. (2014). Behavioral components of impulsivity. Journal of Experimental Psychology: General, 143(2), 850–886. doi:10.1037/a0033981.

    Article  Google Scholar 

  • Stolz, J., Besner, D., & Carr, T. (2005). Implications of measures of reliability for theories of priming: Activity in semantic memory is inherently noisy and uncoordinated. Visual Cognition, 12(2), 284–336.

    Google Scholar 

  • Stumpf, H., Angleitner, A., Wieck, T., Jackson, D., & Beloch-Till, H. (1985). Deutsche personality research form: Hogrefe.

  • Vallesi, A., Canalaz, F., Balestrieri, M., & Brambilla, P. (2015). Modulating speed-accuracy strategies in major depression. Journal of Psychiatric Research, 60, 103–108. doi:10.1016/j.jpsychires.2014.09.017.

    Article  PubMed  Google Scholar 

  • van Ravenzwaaij, D., Donkin, C., & Vandekerckhove, J. (2016). The EZ Diffusion Model Provides a Powerful Test of Simple Empirical Effects. Manuscript submitted for publication.

  • van Ravenzwaaij, D., & Oberauer, K. (2009). How to use the diffusion model: Parameter recovery of three methods: Ez, fast-dm, and DMAT. Journal of Mathematical Psychology, 53(6), 463–473. doi:10.1016/j.jmp.2009.09.004.

    Article  Google Scholar 

  • Vandekerckhove, J., & Tuerlinckx, F. (2007). Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14(6), 1011–1026. doi:10.3758/bf03193087.

    Article  Google Scholar 

  • Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model analysis with MATLAB: A DMAT primer. Behavior Research Methods, 40(1), 61–72. doi:10.3758/brm.40.1.61.

    Article  PubMed  Google Scholar 

  • Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S (4th ed.). New York: Springer.

    Book  Google Scholar 

  • Voss, A., Nagler, M., & Lerche, V. (2013a). Diffusion models in experimental psychology. A practical introduction. Experimental Psychology, 60(6), 385–402.

    Article  PubMed  Google Scholar 

  • Voss, A., Rothermund, K., & Brandtstädter, J. (2008). Interpreting ambiguous stimuli: Separating perceptual and judgmental biases. Journal of Experimental Social Psychology, 44(4), 1048–1056. doi:10.1016/j.jesp.2007.10.009.

    Article  Google Scholar 

  • Voss, A., Rothermund, K., Gast, A., & Wentura, D. (2013b). Cognitive processes in associative and categorical priming: A diffusion model analysis. Journal of Experimental Psychology: General, 142(2), 536.

    Article  Google Scholar 

  • Voss, A., Rothermund, K., & Voss, J. (2004). Interpreting the parameters of the diffusion model: An empirical validation. Memory & Cognition, 32(7), 1206–1220.

    Article  Google Scholar 

  • Voss, A., & Voss, J. (2007). Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39(4), 767–775. doi:10.3758/bf03192967.

    Article  PubMed  Google Scholar 

  • Voss, A., & Voss, J. (2008). A fast numerical algorithm for the estimation of diffusion model parameters. Journal of Mathematical Psychology, 52(1), 1–9.

    Article  Google Scholar 

  • Voss, A., Voss, J., & Klauer, K. C. (2010). Separating response-execution bias from decision bias: Arguments for an additional parameter in Ratcliff’s diffusion model. British Journal of Mathematical and Statistical Psychology, 63(3), 539–555. doi:10.1348/000711009x477581.

    Article  PubMed  Google Scholar 

  • Voss, A., Voss, J., & Lerche, V. (2015). Assessing Cognitive Processes with Diffusion Model Analyses: A Tutorial based on fast-dm-30. Frontiers in Psychology, 6. doi:10.3389/fpsyg.2015.00336.

  • Wagenmakers, E.-J., Grasman, R. P. P. P., & Molenaar, P. C. M. (2005). On the relation between the mean and the variance of a diffusion model response time distribution. Journal of Mathematical Psychology, 49(3), 195–204. doi:10.1016/j.jmp.2005.02.003.

    Article  Google Scholar 

  • Wagenmakers, E.-J., Ratcliff, R., Gomez, P., & McKoon, G. (2008a). A diffusion model account of criterion shifts in the lexical decision task. Journal of Memory and Language, 58(1), 140–159. doi:10.1016/j.jml.2007.04.006.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wagenmakers, E.-J., van der Maas, H. L. J., Dolan, C. V., & Grasman, R. P. P. P. (2008b). EZ does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15(6), 1229–1235. doi:10.3758/pbr.15.6.1229.

    Article  Google Scholar 

  • Wagenmakers, E.-J., van der Maas, H. L. J., & Grasman, R. P. P. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14(1), 3–22. doi:10.3758/bf03194023.

    Article  Google Scholar 

  • Weigard, A., & Huang-Pollock, C. (2014). A diffusion modeling approach to understanding contextual cueing effects in children with ADHD. Journal of Child Psychology and Psychiatry, 55(12), 1336–1344. doi:10.1111/jcpp.12250.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., & Schönfelder, S. (2010). EmoPics: Subjektive und psychophysiologische Evaluation neuen Bildmaterials für die klinisch-biopsychologische Forschung. Zeitschrift für Klinische Psychologie und Psychotherapie, Supplementum, 1(11), 77.

    Google Scholar 

  • White, C. N., Ratcliff, R., Vasey, M., & McKoon, G. (2009). Dysphoria and memory for emotional material: A diffusion-model analysis. Cognition and Emotion, 23(1), 181–205. doi:10.1080/02699930801976770.

    Article  PubMed  PubMed Central  Google Scholar 

  • White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010a). Anxiety enhances threat processing without competition among multiple inputs: A diffusion model analysis. Emotion, 10(5), 662–677. doi:10.1037/a0019474.

    Article  PubMed  Google Scholar 

  • White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010b). Using diffusion models to understand clinical disorders. Journal of Mathematical Psychology, 54(1), 39–52. doi:10.1016/j.jmp.2010.01.004.

    Article  PubMed  PubMed Central  Google Scholar 

  • Whiteside, S. P., & Lynam, D. R. (2001). The five factor model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences, 30(4), 669–689.

    Article  Google Scholar 

  • Wiecki, T. V., Sofer, I., & Frank, M. J. (2013). HDDM: hierarchical bayesian estimation of the drift-diffusion model in python. Frontiers in neuroinformatics, 7.

  • Yap, M. J., Balota, D. A., Sibley, D. E., & Ratcliff, R. (2012). Individual differences in visual word recognition: Insights from the English Lexicon Project. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 53.

    PubMed  Google Scholar 

  • Yap, M. J., Sibley, D. E., Balota, D. A., Ratcliff, R., & Rueckl, J. (2015). Responding to nonwords in the lexical decision task: Insights from the English Lexicon Project. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 597–613. doi:10.1037/xlm0000064.

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Psychologisches Institut, Ruprecht-Karls-Universität Heidelberg, Hauptstrasse 47-51, 69117, Heidelberg, Germany

    Veronika Lerche & Andreas Voss

Authors
  1. Veronika Lerche
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Voss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Veronika Lerche.

Ethics declarations

This research was supported by a Grant from the German Research Foundation to Andreas Voss (Grant No. VO1288/2-1).

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lerche, V., Voss, A. Retest reliability of the parameters of the Ratcliff diffusion model. Psychological Research 81, 629–652 (2017). https://doi.org/10.1007/s00426-016-0770-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00426-016-0770-5

Keywords

Search

Navigation