The role of strategy use in working memory training outcomes,☆☆

https://doi.org/10.1016/j.jml.2019.104064Get rights and content

Highlights

  • Strategies generated during training predict training gains.

  • These self-generated strategies stabilize early on.

  • Externally given strategy elicits only fleeting training task gain.

  • Working memory training represents cognitive skill learning.

Abstract

Cognitive mechanisms underlying the limited transfer effects of working memory (WM) training remain poorly understood. We tested in detail the Strategy Mediation hypothesis, according to which WM training generates task-specific strategies that facilitate performance on the trained task and its untrained variants. This large-scale pre-registered randomized controlled trial (n = 258) used a 4-week adaptive WM training with a single digit n-back task. Strategy use was probed with open-ended strategy reports. We employed a Strategy training group (n = 73) receiving external strategy instruction, a Traditional training group (n = 118) practicing without strategy instruction, and Passive controls (n = 67). Both training groups showed emerging transfer to untrained n-back task variants already at intermediate test after 3 training sessions, extending to all untrained n-back task variants at posttest after 12 training sessions. The Strategy training group outperformed the Traditional training group only at the beginning of training, indicating short-lived strategy manipulation effects. Importantly, in the Traditional training group, strategy evolvement modulated the gains in the trained and untrained n-back tasks, supporting the Strategy Mediation hypothesis. Our results concur with the view of WM training as cognitive skill learning.

Introduction

Introduced almost two decades ago, working memory (WM) training with computerized adaptive tasks initially stirred considerable interest and enthusiasm as the first results suggested rather broad effects for this new cognitive training form (Borella et al., 2010, Brehmer et al., 2012, Chein and Morrison, 2010, Jaeggi et al., 2008, Klingberg et al., 2002). However, besides methodological criticisms (e.g., McCabe et al., 2016, Melby-Lervåg et al., 2016, Morrison and Chein, 2011, Shipstead et al., 2012, Simons et al., 2016; see also Green et al., 2019), the extent of these improvements has been contested. Recent meta-analyses have consistently shown that transfer to other cognitive domains is highly limited, as seen by small effect sizes for far transfer (Au et al., 2015, Karbach and Verhaeghen, 2014, Kassai et al., 2019, Melby-Lervåg and Hulme, 2013, Melby-Lervåg et al., 2016, Sala and Gobet, 2017, Schwaighofer et al., 2015, Weicker et al., 2016). Even within the WM domain, recent meta-analytical evidence indicates that more substantial transfer following training is seen only in WM tasks sharing the same task paradigm with the trained one(s) (Gathercole et al., 2019, Soveri et al., 2017).

All in all, it appears that the very limited transfer following WM training must stem from something else than brain plasticity-based general WM capacity increase that was initially thought to be the driving force behind WM training effects (Klingberg, 2010, Morrison and Chein, 2011, Takeuchi and Kawashima, 2012, von Bastian and Oberauer, 2014). Here we conducted an in-depth analysis of an alternative hypothesis that relates the limited effects of WM training to increased WM efficacy rather than increased capacity (Ericsson & Kintsch, 1995), namely the Strategy Mediation hypothesis (Dunning and Holmes, 2014, Peng and Fuchs, 2017). This hypothesis assumes that WM training leads to adoption of strategies specific to the trained task, leading to performance improvements on the trained task and its untrained variants at posttest (Dunning and Holmes, 2014, McNamara and Scott, 2001). In other words, task-specific strategies that participants adopt during training (for example, starting to group items when training with an adaptive digit span task) can be highly applicable to structurally similar untrained WM tasks as well (e.g., an untrained letter span task), but not necessarily helpful in untrained structurally dissimilar WM tasks (e.g., selective updating). The transfer pattern observed in recent meta-analyses (Gathercole et al., 2019, Soveri et al., 2017) and in most of the methodologically stringent WM training studies (Chooi and Thompson, 2012, Clark et al., 2017, De Simoni and von Bastian, 2018, Guye and von Bastian, 2017, Holmes et al., 2019, Lawlor-Savage and Goghari, 2016, Minear et al., 2016, Sprenger et al., 2013) seems to fit with this hypothesis.

The present study was also prompted by the lack of studies verifying the basic assumptions of the Strategy Mediation hypothesis in the context of ordinary WM training, albeit both spontaneous strategy use (Bailey et al., 2014, De Simoni and von Bastian, 2018, Dunning and Holmes, 2014, McNamara and Scott, 2001) and provision of an external strategy instruction (Borella et al., 2017, Carretti et al., 2007, Peng and Fuchs, 2017, Turley-Ames and Whitfield, 2003) have been related to improved WM task performance. Moreover, most of the studies in which an explicit strategy was provided for the trainees have encompassed very short training periods spanning 1–4 sessions, typically using list recall tasks (Carretti et al., 2007, McNamara and Scott, 2001) or span tasks as the training task (Bailey et al., 2014, Borella et al., 2017, Turley-Ames and Whitfield, 2003).

To our knowledge, only one study has examined how external strategy provision and internally generated strategies affect training outcomes following short-term WM training (Laine, Fellman, Waris, & Nyman, 2018). In the study by Laine et al. (2018), participants practiced with an adaptive single digit n-back task for 30 min. One group was instructed to use a visuospatial strategy in their training session (prompting them to mentally order the incoming digit sequences on top of each other, and to compare whether the upper and lower digit matched or not), whereas another group took the same training session but did not receive any strategy instruction. A third group served as a passive control group. Following intervention, those receiving the visuospatial strategy outperformed the two other groups on two structurally similar untrained tasks (letter n-back, color n-back), whereas no group differences were observed on structurally different untrained WM tasks. Moreover, internally generated strategy use (as measured with level of detail and type of strategy) in the uninstructed groups was related to higher posttest performance on the trained n-back task and its untrained variants. However, as Laine et al. (2018) examined only the effects of very brief (single-session) WM training, it remains open whether their results would hold for typical WM training that spans 4–6 weeks.

With regard to typical, longer-lasting WM training, evidence on the effects of externally provided and internally generated strategies on training outcomes is very limited. Dunning and Holmes (2014) examined internal strategy use following 10 WM training sessions. Their results indicated that the use of strategies (e.g., semantics, visualization, grouping) increased from pre- to posttest in all of their three groups (adaptive WM training, non-adaptive WM training, passive controls). The Strategy Mediation hypothesis would predict strategy changes and concomitant performance improvements particularly in the participants who received training, but Dunning and Holmes (2014) did not find reliable interaction effects between strategy change and pre-post gains in the adaptive training group, possibly due to an underpowered study design.

Recently, De Simoni and von Bastian (2018) ran a large-scale online randomized controlled WM training trial over a 6-week period (with two WM training groups receiving either updating or binding training, and one active control group practicing with a visual search task), and gathered information on strategy use retrospectively. More specifically, they asked their participants whether they had used a strategy during training in their respective training task, and if so, to rate the usefulness of their strategy. While their aim was not to address the issue of strategy development and transfer gains (the strategy reports concerned only the training task), the results showed that the majority of the participants had used one or more strategies during training (updating training group: 83%; binding training group: 88%; active controls: 63%), and that WM trainees reported higher usefulness ratings of their adopted strategies than the active controls.

Lastly, the only longer-term WM training study that examined both the effects of an externally given strategy and internally generated strategies was conducted by Peng and Fuchs (2017). Fifty-eight children at risk of learning problems were randomized to an instructed group receiving WM training together with an explicit rehearsal strategy, a non-instructed group receiving WM training without strategy instruction, or to a passive control group. The two training groups practiced 35 min per day during 10 consecutive school days. Internally generated strategy use was documented by a research assistant during each training session in the non-instructed training group, showing that 28% of those participants employed a self-generated strategy. Amongst the strategy users, the most common strategy was rehearsal (59%), followed by counting (32%), visualization (6%), and semantics (3%). Following training, both training groups showed improved performance on the trained WM tasks, but after controlling for multiple comparisons, neither training group showed transfer to untrained WM tasks or to tasks that assessed listening comprehension and episodic memory. Despite the rigorous collection of internal strategy use across the training period, Peng and Fuchs (2017) did not examine how these self-generated strategies were related to the training and transfer outcomes. Thus, the basic assumptions of the Strategy Mediation hypothesis in the context of ordinary WM training remain untested.

To sum up, the current evidence for the Strategy Mediation hypothesis shows that explicit strategies induce at least short-term performance improvements in the trained task paradigm (Laine et al., 2018), self-generated strategies in ordinary WM training are altered by training (De Simoni and von Bastian, 2018, Dunning and Holmes, 2014, Peng and Fuchs, 2017), and that the training outcomes following instructed WM training are on par with those observed in non-instructed WM training (Peng & Fuchs, 2017). However, to our knowledge, no study has tested the Strategy Mediation hypothesis in a full-blown WM training study by examining the session-by-session evolvement of externally provided vs. internal strategy use, and their predictive value concerning training outcomes.

The present pre-registered (see Supplemental Material) fully online-administered randomized controlled trial sought to test the Strategy Mediation hypothesis in WM training in detail. This hypothesis implicates that exposure to training elicits more strategy use in the trained task. As this strategy adoption is highly task-specific (it concerns only the trained task; cf. Chase and Ericsson, 1982, Lustig et al., 2009, Maguire et al., 2003), practice effects are likely to be seen in the trained task whereas transfer effects are expected to extend only to tasks that are structurally similar to the trained task (Taatgen, 2013). We first documented this very limited transfer, which is familiar from many previous WM training studies, by using the analytical approach typically employed in WM training studies (i.e., baseline comparisons, training improvement, and pre-posttest gains on trained and untrained WM tasks). Then we turned to four more specific predictions concerning the Strategy Mediation hypothesis that we had postulated. All these predictions pertain to actual strategy use as measured by repeated strategy reports collected during testing and training sessions, rather than to the indirect evidence provided by the transfer patterns or retrospective strategy reports collected at posttest only.

  • (i)

    The first prediction was that the characteristics of self-generated strategies significantly predict WM training outcomes on the trained task and its untrained variants. More specifically, repeated practice on the trained WM task should increase strategy use on that task, which in turn would lead to higher performance on the trained task and structurally similar untrained tasks. Following our earlier study (Laine et al., 2018), we measured whether two characteristics of self-reported strategies, namely sophistication of a strategy and the level of detail provided, were related to the predicted training gains. These two interrelated but not fully overlapping strategy measures were treated as continuous variables (cf. Dunlosky and Kane, 2007, McNamara and Scott, 2001). Both of them have been shown to be strongly associated with level of performance on the same WM tasks (Forsberg et al., submitted for publication, Laine et al., 2018). We thus expected that compared to passive controls, the uninstructed training group receiving traditional WM training (henceforth Traditional training group) would report more effective strategy types and more strategic detail over the course of the study, and that within the Traditional training group, these strategy improvements would be linked to higher scores on the trained task and its untrained variants.

  • (ii)

    The second prediction was that provision of an effective external strategy improves subsequent WM training outcomes on the trained task and its untrained variants. This prediction is not directly relevant to traditional WM training where strategy instructions are absent, but it is crucial for establishing that strategy use causally impacts task performance when practicing with WM tasks. Evaluating this potential causal link is also relevant, as WM strategy use has been suggested to be related to high initial WM capacity that enables the allocation of more cognitive resources to strategy deployment (Dunlosky & Kane, 2007). However, facilitation of WM performance by an externally provided training strategy, as compared to trainees not receiving strategy instruction, would speak for strategy use as a cause rather than as an effect. The second prediction was investigated in two ways. First, the training group receiving strategy instruction (henceforth Strategy training group) should improve more on the trained task and its untrained variants than the training group not receiving any strategy instruction. Thus, we went beyond the single-session training analysis by Laine et al. (2018) by examining the transfer effects following three (intermediate test) and twelve (posttest) sessions of WM training coupled with an externally provided strategy. Secondly, we also attempted to directly replicate the findings in the Laine et al. (2018) study by investigating the block-by-block training performance between the two training groups within the first training session. Here we expected that the training task performance of the Strategy training group would surpass that of the Traditional training group that received no strategy instruction.

  • (iii)

    According to our third prediction, self-reported strategies stabilize early on during WM training. This tentative prediction was based on a more general view of WM training, including its effects on strategy deployment, as an instance of cognitive skill learning (Gathercole et al., 2019, Laine et al., 2018). The general theoretical framework for learning a new cognitive skill (e.g., Chein & Schneider, 2012) posits initial executively demanding phases of strategy generation and strategy maintenance, followed by longer-lasting gradual development of task routine. Given that WM tasks are well-structured, we speculated that the strategy generation phase, showing more intra-individual variability in reported strategy use, would be limited to the initial stages of training. Thus, we expected that participants in the Traditional training group would show session-to-session variability in their strategy use particularly in the beginning of training, after which their strategy use would become more stable.

  • (iv)

    The fourth prediction was that the self-generated strategy adopted for the trained task generalizes also to the untrained, structurally similar variants of that task. It seems reasonable to assume that when a well-functioning strategy for the trained WM task has been adopted, the participant may attempt to apply it also to structurally similar tasks. To test this prediction that links performance transfer to strategy transfer, we quantified the overlap in strategy type between the trained n-back task and its three untrained variants for each participant in the Traditional training group and in the Passive control group. If the prediction holds, the former group should show more strategy overlap at intermediate and/or posttest.

To test the four predictions described above, we examined changes in strategy reports and WM performance over the course of the study (baseline test at week 1, intermediate test at week 3, training sessions between week 2–5 for those receiving training, and posttest at week 6) in participants randomized to one of the three different groups. Two of the groups received six hours of WM training (12 × 30 min training sessions of single digit n-back training) over a 4-week period. Thus, the training dose corresponds to the typical amount of WM training employed in previous studies (median 6.67 h of training) in the n-back training meta-analysis by Soveri et al. (2017). One of the training groups practiced with an externally provided strategy (Strategy training group), while the other training group practiced without strategy instructions (the Traditional training group representing the typical WM training condition). The third group was a Passive control group that received no training. The pre-intermediate-posttest battery encompassed the trained WM task (n-back with digits), three untrained variants of the trained task (n-back with letters, colors and locations), as well as six untrained short-term memory and WM tasks that differed structurally from the trained WM task (four variants of the simple span task, two variants of the Running memory task). Using self-reported questionnaires at the end of each assessment point and training session, we registered participants’ WM strategy use on all tasks during the course of the study.

Even though we referred to frequentist statistics in the pre-registration, we chose to employ Bayesian inference throughout our analyses, as they follow the most recent analytical recommendations (Benjamin et al., 2018, Schönbrodt and Wagenmakers, 2018, Wagenmakers et al., 2018), refraining from using p-values due to their shortcomings with regard to replicability and publication bias (e.g., Open Science Collobaration, 2015, Simonsohn et al., 2014). The use of the Bayes factor (BF) has also become more popular in WM training studies during the recent years (e.g., De Simoni and von Bastian, 2018, Gathercole et al., 2019, Guye and von Bastian, 2017, Lawlor-Savage and Goghari, 2016) as it allows researchers to examine the likelihood of their data both under H1 and H0, that is, under the active hypothesis (e.g., strategy use affects training outcomes) and under the corresponding null hypothesis (e.g., strategy effects on training outcomes are absent) (see e.g., Jeffreys, 1961, Kass and Raftery, 1995).

Section snippets

Participants

The participants were 18–50-year-old healthy adults recruited through the crowdworking site Prolific Academic (https://www.prolific.ac/). The inclusion criteria were as follows: English native speakers, no current psychiatric or neurological illnesses, no current use of CNS medication, and no current psychotropic drug use. The study was approved by the Institutional Review Board of the Departments of Psychology and Logopedics, Åbo Akademi University, and it was conducted in accordance with the

Results

Prior to testing the four predictions described earlier, we performed general analyses on the data (see Sections ‘General analyses: Pretest performance of the three groups’, ‘General analyses: Training improvement, motivation, alertness, and expectations’, ‘General analyses: Training outcomes at the intermediate and posttest’, ‘General analyses: Descriptive statistics of strategy use’). The general analyses compared the groups on their pretest performance (all three groups), training

Discussion

The present study set out to the test in detail the Strategy Mediation hypothesis as an underlying cognitive mechanism of WM training. Being linked to the broader theoretical framework of skill learning (Anderson, 1982, Chein and Schneider, 2012, Taatgen, 2013), this hypothesis postulates that training-induced change in WM is driven by compensatory strategies that participants develop during training (Dunning and Holmes, 2014, McNamara and Scott, 2001, Peng and Fuchs, 2017). As strategies are

Acknowledgments

We thank Daniel Wärnå and Janne Hakala for programming the test platform and the cognitive tasks. We are grateful to Claudia von Bastian for her valuable comments on a previous version of this paper.

Funding

ML received funding from the Academy of Finland (Grants No. 260276 and 323251) and the Åbo Akademi University Endowment (grant to the BrainTrain project). DF report grants from Signe and Ane Gyllenberg Foundation.

References (96)

  • S.M. Smith et al.

    A multi-group analysis of online survey respondent data quality: Comparing a regular USA consumer panel to MTurk samples

    Journal of Business Research

    (2016)
  • A.M. Sprenger et al.

    Training working memory: Limits of transfer

    Intelligence

    (2013)
  • R. Stark et al.

    Conditions and effects of example elaboration

    Learning and Instruction

    (2002)
  • K.J. Turley-Ames et al.

    Strategy training and working memory task performance

    Journal of Memory and Language

    (2003)
  • A.E. van’t Veer et al.

    Pre-registration in social psychology—A discussion and suggested template

    Journal of Experimental Social Psychology

    (2016)
  • C.C. von Bastian et al.

    Distinct transfer effects of training different facets of working memory capacity

    Journal of Memory and Language

    (2013)
  • J.R. Anderson

    Acquisition of cognitive skill

    Psychological Review

    (1982)
  • J. Au et al.

    Improving fluid intelligence with training on working memory: A meta-analysis

    Psychonomic Bulletin and Review

    (2015)
  • H. Bailey et al.

    Contribution of strategy use to performance on complex and simple span tasks

    Memory and Cognition

    (2011)
  • H.R. Bailey et al.

    Does strategy training reduce age-related deficits in working memory?

    Gerontology

    (2014)
  • D.J. Benjamin et al.

    Redefine statistical significance

    Nature Human Behaviour

    (2018)
  • T. Bogg et al.

    Reliable gains? Evidence for substantially underpowered designs in studies of working memory training transfer to fluid intelligence

    Frontiers in Psychology

    (2015)
  • E. Borella et al.

    Working memory training for healthy older adults: The role of individual characteristics in explaining short-and long-term gains

    Frontiers in Human Neuroscience

    (2017)
  • E. Borella et al.

    Working memory training in older adults: Evidence of transfer and maintenance effects

    Psychology and Aging

    (2010)
  • E. Borella et al.

    Training working memory in older adults: Is there an advantage of using strategies?

    Psychology and Aging

    (2017)
  • Y. Brehmer et al.

    Working-memory training in younger and older adults: Training gains, transfer, and maintenance

    Frontiers in Human Neuroscience

    (2012)
  • C.N. Bürki et al.

    Individual differences in cognitive plasticity: An investigation of training curves in younger and older adults

    Psychological Research

    (2014)
  • B. Carretti et al.

    Does strategic memory training improve the working memory performance of younger and older adults?

    Experimental Psychology

    (2007)
  • J.M. Chein et al.

    The brain’s learning and control architecture

    Current Directions in Psychological Science

    (2012)
  • J.M. Chein et al.

    Expanding the mind’s workspace: Training and transfer effects with a complex working memory span task

    Psychonomic Bulletin & Review

    (2010)
  • C.M. Clark et al.

    Working memory training in healthy young adults: Support for the null from a randomized comparison to active and passive control groups

    PLoS ONE

    (2017)
  • C. De Simoni et al.

    Working memory updating and binding training: Bayesian evidence supporting the absence of transfer

    Journal of Experimental Psychology: General

    (2018)
  • M.R. Dougherty et al.

    Reevaluating the effectiveness of n-back training on transfer through the Bayesian lens: Support for the null

    Psychonomic Bulletin & Review

    (2016)
  • J. Dunlosky et al.

    The contributions of strategy use to working memory span: A comparison of strategy assessment methods

    Quarterly Journal of Experimental Psychology

    (2007)
  • D.L. Dunning et al.

    Does working memory training promote the use of strategies on untrained working memory tasks?

    Memory and Cognition

    (2014)
  • K.A. Ericsson et al.

    Long-term working memory

    Psychological Review

    (1995)
  • E. Estrada et al.

    A general factor of intelligence fails to account for changes in tests’ scores after cognitive practice: A longitudinal multi-group latent-variable study

    Intelligence

    (2015)
  • D. Fellman et al.

    Training of verbal working memory at sentence level fails to show transfer

    Frontiers in Communication

    (2017)
  • P.C. Fletcher et al.

    Frontal lobes and human memory

    Brain

    (2001)
  • Forsberg, A., Fellman, D., Laine, M., Johnson, W., & Logie, R. H. (submitted for publication). Strategic mediation in...
  • L. Germine et al.

    Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments

    Psychonomic Bulletin & Review

    (2012)
  • S.C. Green et al.

    Improving methodological standards in behavioral interventions for cognitive enhancement

    Journal of Cognitive Enhancement

    (2019)
  • S. Guye et al.

    Working memory training in older adults: Bayesian evidence supporting the absence of transfer

    Psychology and Aging

    (2017)
  • J. Holmes et al.

    Are working memory training effects paradigm-specific?

    Frontiers in Psychology

    (2019)
  • S.M. Jaeggi et al.

    Improving fluid intelligence with training on working memory

    Proceedings of the National Academy of Sciences

    (2008)
  • H. Jeffreys

    The theory of probability

    (1961)
  • I. Juvina et al.

    Modeling control strategies in the N-Back task

  • J. Karbach et al.

    Making working memory work: A meta-analysis of executive-control and working memory training in older adults

    Psychological Science

    (2014)
  • Cited by (38)

    • The domain-specific approach of working memory training

      2022, Developmental Review
      Citation Excerpt :

      Further research should explore the effects of diverse strategy training within the domain-specific approach of WM training (Brod, 2020). Studies with both children and adults suggest that during WM training, individuals in general are prone to use metacognition to find and apply the most efficient task-specific strategies (e.g., Fellman et al., 2020; Jones et al., 2020; Peng & Fuchs, 2017). Thus, combining metacognitive and task-specific strategy training might facilitate WM-academic tasks performance (e.g., students would be directly taught and guided to use metacognitive strategies to monitor task-specific strategies during training).

    View all citing articles on Scopus

    Prior to the data collection, the study protocol was preregistered at AsPredicted.org (https://aspredicted.org/r7qs9.pdf).

    ☆☆

    The data and analysis scripts are available at the Open Science Framework (https://osf.io/q7xe2/).

    View full text