The role of strategy use in working memory training outcomes☆,☆☆
Introduction
Introduced almost two decades ago, working memory (WM) training with computerized adaptive tasks initially stirred considerable interest and enthusiasm as the first results suggested rather broad effects for this new cognitive training form (Borella et al., 2010, Brehmer et al., 2012, Chein and Morrison, 2010, Jaeggi et al., 2008, Klingberg et al., 2002). However, besides methodological criticisms (e.g., McCabe et al., 2016, Melby-Lervåg et al., 2016, Morrison and Chein, 2011, Shipstead et al., 2012, Simons et al., 2016; see also Green et al., 2019), the extent of these improvements has been contested. Recent meta-analyses have consistently shown that transfer to other cognitive domains is highly limited, as seen by small effect sizes for far transfer (Au et al., 2015, Karbach and Verhaeghen, 2014, Kassai et al., 2019, Melby-Lervåg and Hulme, 2013, Melby-Lervåg et al., 2016, Sala and Gobet, 2017, Schwaighofer et al., 2015, Weicker et al., 2016). Even within the WM domain, recent meta-analytical evidence indicates that more substantial transfer following training is seen only in WM tasks sharing the same task paradigm with the trained one(s) (Gathercole et al., 2019, Soveri et al., 2017).
All in all, it appears that the very limited transfer following WM training must stem from something else than brain plasticity-based general WM capacity increase that was initially thought to be the driving force behind WM training effects (Klingberg, 2010, Morrison and Chein, 2011, Takeuchi and Kawashima, 2012, von Bastian and Oberauer, 2014). Here we conducted an in-depth analysis of an alternative hypothesis that relates the limited effects of WM training to increased WM efficacy rather than increased capacity (Ericsson & Kintsch, 1995), namely the Strategy Mediation hypothesis (Dunning and Holmes, 2014, Peng and Fuchs, 2017). This hypothesis assumes that WM training leads to adoption of strategies specific to the trained task, leading to performance improvements on the trained task and its untrained variants at posttest (Dunning and Holmes, 2014, McNamara and Scott, 2001). In other words, task-specific strategies that participants adopt during training (for example, starting to group items when training with an adaptive digit span task) can be highly applicable to structurally similar untrained WM tasks as well (e.g., an untrained letter span task), but not necessarily helpful in untrained structurally dissimilar WM tasks (e.g., selective updating). The transfer pattern observed in recent meta-analyses (Gathercole et al., 2019, Soveri et al., 2017) and in most of the methodologically stringent WM training studies (Chooi and Thompson, 2012, Clark et al., 2017, De Simoni and von Bastian, 2018, Guye and von Bastian, 2017, Holmes et al., 2019, Lawlor-Savage and Goghari, 2016, Minear et al., 2016, Sprenger et al., 2013) seems to fit with this hypothesis.
The present study was also prompted by the lack of studies verifying the basic assumptions of the Strategy Mediation hypothesis in the context of ordinary WM training, albeit both spontaneous strategy use (Bailey et al., 2014, De Simoni and von Bastian, 2018, Dunning and Holmes, 2014, McNamara and Scott, 2001) and provision of an external strategy instruction (Borella et al., 2017, Carretti et al., 2007, Peng and Fuchs, 2017, Turley-Ames and Whitfield, 2003) have been related to improved WM task performance. Moreover, most of the studies in which an explicit strategy was provided for the trainees have encompassed very short training periods spanning 1–4 sessions, typically using list recall tasks (Carretti et al., 2007, McNamara and Scott, 2001) or span tasks as the training task (Bailey et al., 2014, Borella et al., 2017, Turley-Ames and Whitfield, 2003).
To our knowledge, only one study has examined how external strategy provision and internally generated strategies affect training outcomes following short-term WM training (Laine, Fellman, Waris, & Nyman, 2018). In the study by Laine et al. (2018), participants practiced with an adaptive single digit n-back task for 30 min. One group was instructed to use a visuospatial strategy in their training session (prompting them to mentally order the incoming digit sequences on top of each other, and to compare whether the upper and lower digit matched or not), whereas another group took the same training session but did not receive any strategy instruction. A third group served as a passive control group. Following intervention, those receiving the visuospatial strategy outperformed the two other groups on two structurally similar untrained tasks (letter n-back, color n-back), whereas no group differences were observed on structurally different untrained WM tasks. Moreover, internally generated strategy use (as measured with level of detail and type of strategy) in the uninstructed groups was related to higher posttest performance on the trained n-back task and its untrained variants. However, as Laine et al. (2018) examined only the effects of very brief (single-session) WM training, it remains open whether their results would hold for typical WM training that spans 4–6 weeks.
With regard to typical, longer-lasting WM training, evidence on the effects of externally provided and internally generated strategies on training outcomes is very limited. Dunning and Holmes (2014) examined internal strategy use following 10 WM training sessions. Their results indicated that the use of strategies (e.g., semantics, visualization, grouping) increased from pre- to posttest in all of their three groups (adaptive WM training, non-adaptive WM training, passive controls). The Strategy Mediation hypothesis would predict strategy changes and concomitant performance improvements particularly in the participants who received training, but Dunning and Holmes (2014) did not find reliable interaction effects between strategy change and pre-post gains in the adaptive training group, possibly due to an underpowered study design.
Recently, De Simoni and von Bastian (2018) ran a large-scale online randomized controlled WM training trial over a 6-week period (with two WM training groups receiving either updating or binding training, and one active control group practicing with a visual search task), and gathered information on strategy use retrospectively. More specifically, they asked their participants whether they had used a strategy during training in their respective training task, and if so, to rate the usefulness of their strategy. While their aim was not to address the issue of strategy development and transfer gains (the strategy reports concerned only the training task), the results showed that the majority of the participants had used one or more strategies during training (updating training group: 83%; binding training group: 88%; active controls: 63%), and that WM trainees reported higher usefulness ratings of their adopted strategies than the active controls.
Lastly, the only longer-term WM training study that examined both the effects of an externally given strategy and internally generated strategies was conducted by Peng and Fuchs (2017). Fifty-eight children at risk of learning problems were randomized to an instructed group receiving WM training together with an explicit rehearsal strategy, a non-instructed group receiving WM training without strategy instruction, or to a passive control group. The two training groups practiced 35 min per day during 10 consecutive school days. Internally generated strategy use was documented by a research assistant during each training session in the non-instructed training group, showing that 28% of those participants employed a self-generated strategy. Amongst the strategy users, the most common strategy was rehearsal (59%), followed by counting (32%), visualization (6%), and semantics (3%). Following training, both training groups showed improved performance on the trained WM tasks, but after controlling for multiple comparisons, neither training group showed transfer to untrained WM tasks or to tasks that assessed listening comprehension and episodic memory. Despite the rigorous collection of internal strategy use across the training period, Peng and Fuchs (2017) did not examine how these self-generated strategies were related to the training and transfer outcomes. Thus, the basic assumptions of the Strategy Mediation hypothesis in the context of ordinary WM training remain untested.
To sum up, the current evidence for the Strategy Mediation hypothesis shows that explicit strategies induce at least short-term performance improvements in the trained task paradigm (Laine et al., 2018), self-generated strategies in ordinary WM training are altered by training (De Simoni and von Bastian, 2018, Dunning and Holmes, 2014, Peng and Fuchs, 2017), and that the training outcomes following instructed WM training are on par with those observed in non-instructed WM training (Peng & Fuchs, 2017). However, to our knowledge, no study has tested the Strategy Mediation hypothesis in a full-blown WM training study by examining the session-by-session evolvement of externally provided vs. internal strategy use, and their predictive value concerning training outcomes.
The present pre-registered (see Supplemental Material) fully online-administered randomized controlled trial sought to test the Strategy Mediation hypothesis in WM training in detail. This hypothesis implicates that exposure to training elicits more strategy use in the trained task. As this strategy adoption is highly task-specific (it concerns only the trained task; cf. Chase and Ericsson, 1982, Lustig et al., 2009, Maguire et al., 2003), practice effects are likely to be seen in the trained task whereas transfer effects are expected to extend only to tasks that are structurally similar to the trained task (Taatgen, 2013). We first documented this very limited transfer, which is familiar from many previous WM training studies, by using the analytical approach typically employed in WM training studies (i.e., baseline comparisons, training improvement, and pre-posttest gains on trained and untrained WM tasks). Then we turned to four more specific predictions concerning the Strategy Mediation hypothesis that we had postulated. All these predictions pertain to actual strategy use as measured by repeated strategy reports collected during testing and training sessions, rather than to the indirect evidence provided by the transfer patterns or retrospective strategy reports collected at posttest only.
- (i)
The first prediction was that the characteristics of self-generated strategies significantly predict WM training outcomes on the trained task and its untrained variants. More specifically, repeated practice on the trained WM task should increase strategy use on that task, which in turn would lead to higher performance on the trained task and structurally similar untrained tasks. Following our earlier study (Laine et al., 2018), we measured whether two characteristics of self-reported strategies, namely sophistication of a strategy and the level of detail provided, were related to the predicted training gains. These two interrelated but not fully overlapping strategy measures were treated as continuous variables (cf. Dunlosky and Kane, 2007, McNamara and Scott, 2001). Both of them have been shown to be strongly associated with level of performance on the same WM tasks (Forsberg et al., submitted for publication, Laine et al., 2018). We thus expected that compared to passive controls, the uninstructed training group receiving traditional WM training (henceforth Traditional training group) would report more effective strategy types and more strategic detail over the course of the study, and that within the Traditional training group, these strategy improvements would be linked to higher scores on the trained task and its untrained variants.
- (ii)
The second prediction was that provision of an effective external strategy improves subsequent WM training outcomes on the trained task and its untrained variants. This prediction is not directly relevant to traditional WM training where strategy instructions are absent, but it is crucial for establishing that strategy use causally impacts task performance when practicing with WM tasks. Evaluating this potential causal link is also relevant, as WM strategy use has been suggested to be related to high initial WM capacity that enables the allocation of more cognitive resources to strategy deployment (Dunlosky & Kane, 2007). However, facilitation of WM performance by an externally provided training strategy, as compared to trainees not receiving strategy instruction, would speak for strategy use as a cause rather than as an effect. The second prediction was investigated in two ways. First, the training group receiving strategy instruction (henceforth Strategy training group) should improve more on the trained task and its untrained variants than the training group not receiving any strategy instruction. Thus, we went beyond the single-session training analysis by Laine et al. (2018) by examining the transfer effects following three (intermediate test) and twelve (posttest) sessions of WM training coupled with an externally provided strategy. Secondly, we also attempted to directly replicate the findings in the Laine et al. (2018) study by investigating the block-by-block training performance between the two training groups within the first training session. Here we expected that the training task performance of the Strategy training group would surpass that of the Traditional training group that received no strategy instruction.
- (iii)
According to our third prediction, self-reported strategies stabilize early on during WM training. This tentative prediction was based on a more general view of WM training, including its effects on strategy deployment, as an instance of cognitive skill learning (Gathercole et al., 2019, Laine et al., 2018). The general theoretical framework for learning a new cognitive skill (e.g., Chein & Schneider, 2012) posits initial executively demanding phases of strategy generation and strategy maintenance, followed by longer-lasting gradual development of task routine. Given that WM tasks are well-structured, we speculated that the strategy generation phase, showing more intra-individual variability in reported strategy use, would be limited to the initial stages of training. Thus, we expected that participants in the Traditional training group would show session-to-session variability in their strategy use particularly in the beginning of training, after which their strategy use would become more stable.
- (iv)
The fourth prediction was that the self-generated strategy adopted for the trained task generalizes also to the untrained, structurally similar variants of that task. It seems reasonable to assume that when a well-functioning strategy for the trained WM task has been adopted, the participant may attempt to apply it also to structurally similar tasks. To test this prediction that links performance transfer to strategy transfer, we quantified the overlap in strategy type between the trained n-back task and its three untrained variants for each participant in the Traditional training group and in the Passive control group. If the prediction holds, the former group should show more strategy overlap at intermediate and/or posttest.
To test the four predictions described above, we examined changes in strategy reports and WM performance over the course of the study (baseline test at week 1, intermediate test at week 3, training sessions between week 2–5 for those receiving training, and posttest at week 6) in participants randomized to one of the three different groups. Two of the groups received six hours of WM training (12 × 30 min training sessions of single digit n-back training) over a 4-week period. Thus, the training dose corresponds to the typical amount of WM training employed in previous studies (median 6.67 h of training) in the n-back training meta-analysis by Soveri et al. (2017). One of the training groups practiced with an externally provided strategy (Strategy training group), while the other training group practiced without strategy instructions (the Traditional training group representing the typical WM training condition). The third group was a Passive control group that received no training. The pre-intermediate-posttest battery encompassed the trained WM task (n-back with digits), three untrained variants of the trained task (n-back with letters, colors and locations), as well as six untrained short-term memory and WM tasks that differed structurally from the trained WM task (four variants of the simple span task, two variants of the Running memory task). Using self-reported questionnaires at the end of each assessment point and training session, we registered participants’ WM strategy use on all tasks during the course of the study.
Even though we referred to frequentist statistics in the pre-registration, we chose to employ Bayesian inference throughout our analyses, as they follow the most recent analytical recommendations (Benjamin et al., 2018, Schönbrodt and Wagenmakers, 2018, Wagenmakers et al., 2018), refraining from using p-values due to their shortcomings with regard to replicability and publication bias (e.g., Open Science Collobaration, 2015, Simonsohn et al., 2014). The use of the Bayes factor (BF) has also become more popular in WM training studies during the recent years (e.g., De Simoni and von Bastian, 2018, Gathercole et al., 2019, Guye and von Bastian, 2017, Lawlor-Savage and Goghari, 2016) as it allows researchers to examine the likelihood of their data both under H1 and H0, that is, under the active hypothesis (e.g., strategy use affects training outcomes) and under the corresponding null hypothesis (e.g., strategy effects on training outcomes are absent) (see e.g., Jeffreys, 1961, Kass and Raftery, 1995).
Section snippets
Participants
The participants were 18–50-year-old healthy adults recruited through the crowdworking site Prolific Academic (https://www.prolific.ac/). The inclusion criteria were as follows: English native speakers, no current psychiatric or neurological illnesses, no current use of CNS medication, and no current psychotropic drug use. The study was approved by the Institutional Review Board of the Departments of Psychology and Logopedics, Åbo Akademi University, and it was conducted in accordance with the
Results
Prior to testing the four predictions described earlier, we performed general analyses on the data (see Sections ‘General analyses: Pretest performance of the three groups’, ‘General analyses: Training improvement, motivation, alertness, and expectations’, ‘General analyses: Training outcomes at the intermediate and posttest’, ‘General analyses: Descriptive statistics of strategy use’). The general analyses compared the groups on their pretest performance (all three groups), training
Discussion
The present study set out to the test in detail the Strategy Mediation hypothesis as an underlying cognitive mechanism of WM training. Being linked to the broader theoretical framework of skill learning (Anderson, 1982, Chein and Schneider, 2012, Taatgen, 2013), this hypothesis postulates that training-induced change in WM is driven by compensatory strategies that participants develop during training (Dunning and Holmes, 2014, McNamara and Scott, 2001, Peng and Fuchs, 2017). As strategies are
Acknowledgments
We thank Daniel Wärnå and Janne Hakala for programming the test platform and the cognitive tasks. We are grateful to Claudia von Bastian for her valuable comments on a previous version of this paper.
Funding
ML received funding from the Academy of Finland (Grants No. 260276 and 323251) and the Åbo Akademi University Endowment (grant to the BrainTrain project). DF report grants from Signe and Ane Gyllenberg Foundation.
References (96)
- et al.
Mixed-effects modeling with crossed random effects for subjects and items
Journal of Memory and Language
(2008) - et al.
Skill and working memory
Psychology of Learning and Motivation – Advances in Research and Theory
(1982) - et al.
Working memory training does not improve intelligence in healthy young adults
Intelligence
(2012) - et al.
Maintenance versus manipulation of information held in working memory: An event-related fMRI study
Brain and Cognition
(1999) - et al.
Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English
Personality and Individual Differences
(2015) - et al.
The reading span test and its predictive power for reading comprehension ability
Journal of Memory and Language
(2004) - et al.
Working memory training involves learning new skills
Journal of Memory and Language
(2019) Training and plasticity of working memory
Trends in Cognitive Sciences
(2010)- et al.
The Stroop effect on the internet
Computers in Human Behavior
(2006) - et al.
Learning from worked-out examples: The effects of example variability and elicited self-explanations
Contemporary Educational Psychology
(1998)
A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples
Journal of Business Research
Training working memory: Limits of transfer
Intelligence
Conditions and effects of example elaboration
Learning and Instruction
Strategy training and working memory task performance
Journal of Memory and Language
Pre-registration in social psychology—A discussion and suggested template
Journal of Experimental Social Psychology
Distinct transfer effects of training different facets of working memory capacity
Journal of Memory and Language
Acquisition of cognitive skill
Psychological Review
Improving fluid intelligence with training on working memory: A meta-analysis
Psychonomic Bulletin and Review
Contribution of strategy use to performance on complex and simple span tasks
Memory and Cognition
Does strategy training reduce age-related deficits in working memory?
Gerontology
Redefine statistical significance
Nature Human Behaviour
Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence
Frontiers in Psychology
Working memory training for healthy older adults: The role of individual characteristics in explaining short-and long-term gains
Frontiers in Human Neuroscience
Working memory training in older adults: Evidence of transfer and maintenance effects
Psychology and Aging
Training working memory in older adults: Is there an advantage of using strategies?
Psychology and Aging
Working-memory training in younger and older adults: Training gains, transfer, and maintenance
Frontiers in Human Neuroscience
Individual differences in cognitive plasticity: An investigation of training curves in younger and older adults
Psychological Research
Does strategic memory training improve the working memory performance of younger and older adults?
Experimental Psychology
The brain’s learning and control architecture
Current Directions in Psychological Science
Expanding the mind’s workspace: Training and transfer effects with a complex working memory span task
Psychonomic Bulletin & Review
Working memory training in healthy young adults: Support for the null from a randomized comparison to active and passive control groups
PLoS ONE
Working memory updating and binding training: Bayesian evidence supporting the absence of transfer
Journal of Experimental Psychology: General
Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null
Psychonomic Bulletin & Review
The contributions of strategy use to working memory span: A comparison of strategy assessment methods
Quarterly Journal of Experimental Psychology
Does working memory training promote the use of strategies on untrained working memory tasks?
Memory and Cognition
Long-term working memory
Psychological Review
A general factor of intelligence fails to account for changes in tests’ scores after cognitive practice: A longitudinal multi-group latent-variable study
Intelligence
Training of verbal working memory at sentence level fails to show transfer
Frontiers in Communication
Frontal lobes and human memory
Brain
Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments
Psychonomic Bulletin & Review
Improving methodological standards in behavioral interventions for cognitive enhancement
Journal of Cognitive Enhancement
Working memory training in older adults: Bayesian evidence supporting the absence of transfer
Psychology and Aging
Are working memory training effects paradigm-specific?
Frontiers in Psychology
Improving fluid intelligence with training on working memory
Proceedings of the National Academy of Sciences
The theory of probability
Modeling control strategies in the N-Back task
Making working memory work: A meta-analysis of executive-control and working memory training in older adults
Psychological Science
Cited by (38)
The domain-specific approach of working memory training
2022, Developmental ReviewCitation Excerpt :Further research should explore the effects of diverse strategy training within the domain-specific approach of WM training (Brod, 2020). Studies with both children and adults suggest that during WM training, individuals in general are prone to use metacognition to find and apply the most efficient task-specific strategies (e.g., Fellman et al., 2020; Jones et al., 2020; Peng & Fuchs, 2017). Thus, combining metacognitive and task-specific strategy training might facilitate WM-academic tasks performance (e.g., students would be directly taught and guided to use metacognitive strategies to monitor task-specific strategies during training).
Unicorn, Hare, or Tortoise? Using Machine Learning to Predict Working Memory Training Performance
2023, Journal of CognitionSelf-reported strategy use in working memory tasks
2024, Scientific ReportsSpontaneous memory strategies in a videogame simulating everyday memory tasks
2024, Quarterly Journal of Experimental Psychology
- ☆
Prior to the data collection, the study protocol was preregistered at AsPredicted.org (https://aspredicted.org/r7qs9.pdf).
- ☆☆
The data and analysis scripts are available at the Open Science Framework (https://osf.io/q7xe2/).