Elsevier

Early Childhood Research Quarterly

Volume 44, 3rd Quarter 2018, Pages 15-23
Early Childhood Research Quarterly

A head-to-toes approach to computerized testing of executive functioning in young children

https://doi.org/10.1016/j.ecresq.2018.01.008Get rights and content

Highlights

  • Findings support the use of the computerized task (HTKS-c) as a measure of IC in preschool-age children.

  • Standardizing administration allowed for a more accurate measurement of IC.

  • Both versions of the HTKS task showed similar relations with measures of other developmental constructs.

  • The computerized version of the HTKS may extend the age range with which the original task can be used.

Abstract

The Head-Toes-Knees-Shoulders (HTKS) task is a commonly used measure of executive functioning (EF) designed for use with young children. Substantial associations between the HTKS task and early academic skills and behavioral outcomes have consistently been reported. Although the HTKS task is a useful measure of EF for young children, the original version of the task has a number of limitations, including ceiling effects toward the end of kindergarten, potential motivational issues with older children, and the possibility of errors in administration and scoring due to the live administration of the task. Lonigan (2013) developed a computerized version of the HTKS task (HTKS-c) as a means of addressing these issues. This study examined the construct, convergent, and divergent validity of the HTKS-c task with important developmental outcomes, to determine its utility as a measure of EF for use with young children. In the current study, 126 preschool-aged children were administered both versions of the HTKS as well as a battery of performance-based EF tasks, measures of pre-literacy and mathematics skills, and a delay task. Parents and teachers completed report-based measures of externalizing behaviors. Findings indicated that the HTKS-c task offered a valid measure of EF, equivalent to that of the original HTKS task, for use with preschool children. Furthermore, the HTKS-c offered the added benefits of standardized administration, more accurate scoring of self-corrected responses, and, although not examined in this study, the capability of response-latency scoring in addition to accuracy scoring.

Introduction

Self-regulation is an important developmental capacity related to other significant developmental outcomes, including academic and social difficulties and problem behaviors (e.g., Blair, 2002; Eisenberg et al., 2000; Lonigan, Allan, & Phillips, 2017; Lonigan, Spiegel et al., 2017; Vitaro, Brendgen, Larose, & Trembaly, 2005). The construct of self-regulation is broadly defined as consisting of multiple aspects of children’s behavior, including planning, directing, and controlling abilities (e.g., Mahone & Hoffman, 2007). More narrowly, self-regulation can be defined as the partially separable constructs of executive function (EF), attentional control, and effortful control (Allan, Allan, Lerner, Farrington, & Lonigan, 2015; Blair, Urasche, Greenberg, Vernon-Feagans, & Family Life Project Investigators, 2015). From a cognitive perspective, the construct most frequently associated with self-regulation is EF; EF is a multidimensional construct defined by domain-general cognitive processes that are associated with the regulation of emotions and goal-directed behaviors. Specifically, EF is hypothesized to consist of three distinct components: inhibitory control (IC), working memory (WM), and shifting (SH; Miyake, Friedman, Emerson, Witzki, & Howerter, 2000).

EF skills, including IC and WM, are associated with academic outcomes in preschool and early elementary school (e.g., Allan & Lonigan, 2011; Blair & Razza, 2007; Lonigan, Allan et al., 2017; McClelland et al., 2007). With young children, these EF skills have been shown to be longitudinally predictive of academic abilities independent of initial cognitive (e.g., McClelland, Morrison, & Holmes, 2000) and academic skills (e.g., Howse, Lange, Farran, & Boyles, 2003; McClelland et al., 2000; Ponitz, McClelland, Matthews, & Morrison, 2009). Additionally, children with poor EF are at an increased risk of academic difficulties and school dropout in late childhood and early adolescence (Duncan et al., 2007; Eisenberg et al., 2000; Vitaro et al., 2005). Results of meta-analytic studies examining the relations between EF and externalizing behavior disorders, demonstrate moderate associations between self-regulation and symptoms of Attention-Deficit/Hyperactivity Disorder (ADHD; Alderson, Rapport, & Kofler, 2007; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005) and somewhat inconsistent relations with symptoms of Oppositional Defiant Disorder (ODD) and Conduct Disorder (Pennington & Ozonoff, 1996). Among non-clinical populations of young children, EF has also been shown to be associated with individual differences in externalizing behaviors (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017; Hughes & Ensor, 2011; Rezazadeh, Wilding, & Cornish, 2011; Sulik, Blair, Mills-Koonce, Berry, Greenberg, & The Family Life Project Investigators, 2015). Given the substantial associations between EF and academic and behavioral functioning in young children, the accurate assessment of EF skills may be particularly important for the identification of children at risk for difficulties upon school entry.

Despite the fact that EF skills have consistently been found to relate to a wide array of important developmental outcomes (e.g., McClelland et al., 2000; Willcutt et al., 2005), only minor associations have been found between EF skills and children’s ability to delay rewards (e.g., Allan & Lonigan, 2011). Direct comparisons of delay aversion and IC tasks among children with ADHD indicate that these constructs represent distinct influences on child psychopathology (see Sonuga-Barke, 2002, for a review). Additionally, the nonsignificant negative correlations between delay aversion and IC tasks reported in previous studies of community-based samples of children (e.g., Allan & Lonigan, 2011) offer further support that delay aversion tasks measure a different underlying construct than IC.

Young children’s self-regulation and EF skills are assessed via either report- or performance-based measures. Report-based measures involve self-, parent-, or teacher-ratings of EF skills based on observations of behaviors within everyday situations. Report-based measures of EF offer an easy to administer, time efficient, and ecologically valid alternative to performance-based measures. However, such measures have been criticized as assessing skills only tangentially related to executive functioning (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017; Spiegel, Lonigan, & Phillips, 2017), and, although they are designed to assess the same underlying construct as performance-based measures, the two types of measures often demonstrate only small to moderate associations (Toplak, West, & Stanovich, 2013).

Performance-based measures offer a direct assessment of EF skills. One such measure is the Head-to-Toes task (HTT; Ponitz et al., 2008). The HTT task is an adapted version of the Head-to-Feet assessment (McCabe, Rebello-Britto, Hernandez, & Brooks-Gunn, 2004). The HTT task is a response-conflict EF task (i.e., a task that requires the child to perform a subordinate response while refraining from performing a dominant response) that is played as a game in which participants are told to do the opposite of a command. For instance, participants are told that when they hear the command, “touch your toes” they should touch their head. A more complex version of the HTT task, the Head-Toes-Knees-Shoulders task (HTKS; Ponitz et al., 2009) adds two additional commands. Specifically, participants are told to touch their knees when they hear the command “touch your shoulders” and touch their shoulders when they hear the command “touch your knees.” The head and toes commands are interspersed for the first 10 trials and all four commands are interspersed for the second 10 trials. Although the HTT and HTKS tasks are primarily measures of IC, other self-regulatory skills, such as WM and attention, may also be measured (Ponitz et al., 2008).

The HTKS task is easy to administer and requires few materials and a minimal time commitment. In addition, the HTKS task is designed to be appealing to young children, incorporating aspects of commonly played games with which children are likely to be familiar. Overall, the HTT and HTKS tasks are ideal measures for administration in both the laboratory and field settings and promote task engagement when utilized with young children. Both HTT and HTKS tasks demonstrate strong construct validity, with moderate relations reported between scores on these tasks and performance- and report-based measures of self-regulation (e.g., Allan & Lonigan, 2011; Graziano et al., 2015). Specifically, children with higher performance on the HTKS were rated as having higher regulatory skills by parents (Graziano et al., 2015) and teachers (Ponitz et al., 2009) and as demonstrating stronger EF abilities on other performance-based measures of IC (e.g., Allan & Lonigan, 2011; Fuhs, Farran, & Nesbitt, 2015) and WM (e.g., Graziano et al., 2015; Lonigan, Lerner, Goodrich, Farrington, & Allan, 2016). Consistent with the broader literature on EF, results of studies demonstrate significant concurrent and longitudinal associations between scores on the HTT and HTKS tasks and academic and behavioral outcomes when used with preschool and kindergarten children. Significant positive associations are consistently reported between scores on the HTKS task and measures of literacy (e.g., Allan & Lonigan, 2011; Graziano et al., 2015; Matthews, Ponitz, & Morrison, 2009; McClelland et al., 2007) and mathematics (e.g., Ponitz et al., 2009; Skibbe, Connor, Morrison, & Jewkes, 2011, Wanless et al., 2011) both concurrently and longitudinally. Substantial associations have also been found between the HTKS task and report-based measures of externalizing behavior problems; however, results seem to be discrepant based on the specific symptoms examined (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017).

Although the HTT and HTKS tasks are useful measures of EF for preschool and kindergarten children, the age range within which they can be reliably utilized may be limited. Results of studies that have used the HTT or HTKS tasks indicate that by kindergarten many children are able to perform these tasks without errors (Ponitz et al., 2008), resulting in ceiling effects and non-normally distributed scores. Results indicate that the HTT and HTKS tasks are most useful and appropriate when used with children prior to kindergarten (Ponitz et al., 2009). Ceiling effects may be especially pronounced in girls, with studies consistently reporting significantly less variability in HTKS scores for girls than for boys (Matthews et al., 2009; Wanless et al., 2013).

An important question about children’s EF concerns the rate and timing of development of EF capacity. Important developments in EF skills occur throughout the first nine years of life, with marked changes occurring between the ages of 3 and 6 (e.g., Diamond & Taylor, 1996; Espy, Kaufmann, McDiarmid, & Glisky, 1999) and between ages 7 and 9 (Anderson, 2002). Optimally, measures of EF would span the age range during which there is rapid growth of EF capacity; however, ceiling effects on the HTT and HTKS tasks (e.g., Matthews et al., 2009) limit the ability to use these measures longitudinally. Moreover, whereas younger children are likely to find the HTT and HTKS tasks fun and game like, it is possible that such tasks are less developmentally appropriate for older children, potentially resulting in suboptimal performance when older children are unmotivated to perform well on a task that they find age-inappropriate. Such distributional problems and motivational issues limit the ability to use these tasks to study the development and developmental significance of EF skills beyond kindergarten.

As a means of addressing the potential problems associated with the HTKS task, Lonigan (2013) developed a computerized version of the HTKS task. Previous research on other neuropsychological performance tasks has similarly addressed distributional and motivational concerns by creating adapted computerized versions of tasks to more accurately detect performance errors (i.e., incorrect or self-corrected responses) by standardizing test administration and providing automated immediate response recording (Luciana, 2003). To the best of our knowledge, no study to date has examined task motivation of children on computerized versus standard task administration; however, findings from studies with adults indicate a strong preference towards computerized task administration (Noyes & Garland, 2008). In the computerized version of the HTKS, participants are presented with an outline of a human body. Just as with the original task, the computerized version requires participants to remember two commands for the first 10 trials and four commands for the second 10 trials. Like the original HTKS task, the computerized version is a response-conflict task, meaning that throughout the task participants are required to do the opposite of each command.

Although there are numerous advantages to using the computerized version of the HTKS task in older children, younger children may have difficulties with the computerized version because computerizing task administration may increase task demands for reasons that are independent of EF skills (e.g., variable knowledge of computers and fine motor skills needed to respond with a mouse; Saçkes, Trundle, & Bell, 2011). Given the strength of results to date for the original HTT and HTKS tasks with younger children, it is important to demonstrate that both the original and computerized versions assess the same underlying construct to the same degree within this age group. Before the longitudinal applications of the computerized HTKS can be examined it is important to first examine the utility of the measure with the children it was originally designed for use with. Therefore, the primary goal of this study was to evaluate the degree to which the original and computerized versions of the HTKS task provided an equivalent measurement of EF for young children. It was hypothesized that the original and computerized versions of the HTKS task would provide equivalent measures of EF in preschool-age children. Children were administered a battery of commonly used performance-based EF, delay, and academic ability tasks, and parents and teachers completed measures of externalizing behaviors. Relations between the computerized HTKS task and these measures were hypothesized to be similar to those previously reported between the HTT and HTKS tasks and similar measures of important developmental outcomes.

Section snippets

Participants

Data from 130 preschool children were collected for this study. Because young children may not be familiar with computer tasks or may have less well-developed fine motor skills that could influence the degree to which their recorded responses reflect the underlying construct of IC, a mouse-tracking task was administered to assess participants’ abilities to manipulate and point with a mouse. Four children were excluded because they were unable to demonstrate competency using a computer mouse or

Results

Descriptive statistics for all measures are shown in Table 1. The majority of children’s scores on the original HTKS fell above the halfway score of 20 out of 40 points (i.e., 60.8%), with 36% of children scoring at or above 30. However, despite the majority of children’s scores falling above the halfway score, examination of the skewness and kurtosis indicated a normal distribution or scores, consistent with findings from Ponitz et al. (2009). Scores on the computerized HTKS task were also

Discussion

The purpose of this study was to examine whether the HTKS-c measured the same underlying construct as the original HTKS task by examining the convergent and divergent validity of these tasks with measures of important developmental outcomes. Overall, the computerized version demonstrated good construct validity when compared to the original HTKS task and other commonly used measures of IC. Further, the HTKS-c demonstrated good utility and convergent validity as evidenced by substantial

Conclusions

Previous studies have examined the psychometric properties and utility of the original HTKS task as a measure of IC in young children (e.g., Allan & Lonigan 2011; Graziano et al., 2015; Ponitz et al., 2009). However, the original version of the task has a number of limitations, including ceiling effects toward the end of kindergarten, motivational issues with older children, and the possibility of errors in administration and scoring during live administration. The computerized version of the

Acknowledgments

This research and report was supported by grants from the Institute of Education Sciences, US Department of Education (R305B090021) and the Eunice Kennedy Schriver National Institute of Child Health and Human Development (HD052120). The views expressed herein are those of the authors and have not been reviewed or approved by the granting agency.

References (62)

  • L.E. Skibbe et al.

    Schooling effects on preschoolers’ self-regulation, early literacy, and language growth

    Early Childhood Research Quarterly

    (2011)
  • R. Smith-Donald et al.

    Preliminary construct and concurrent validity of the preschool self-regulation assessment (PSRA) for field-based research

    Early Childhood Research Quarterly

    (2007)
  • E.J. Sonuga-Barke

    Psychological heterogeneity in AD/HD—A dual pathway model of behaviour and cognition

    Behavioural Brain Research

    (2002)
  • S.B. Wanless et al.

    Gender differences in behavioral regulation in four societies: The United States, Taiwan, South Korea, and China

    Early Childhood Research Quarterly

    (2013)
  • E.G. Willcutt et al.

    Validity of the executive function theory of Attention-Deficit/Hyperactivity Disorder: A meta-analytic review

    Biological Psychiatry

    (2005)
  • R.M. Alderson et al.

    Attention-Deficit/Hyperactivity Disorder and behavioral inhibition: A meta-analytic review of the stop-signal paradigm

    Journal of Abnormal Child Psychology

    (2007)
  • N.P. Allan et al.

    Examining the dimensionality of effortful control in preschool children and its relation to academic and socioemotional indicators

    Developmental Psychology

    (2011)
  • N.P. Allan et al.

    Relations between inhibitory control and the development of academic skills in preschool and kindergarten: A meta-analysis

    Developmental Psychology

    (2014)
  • American Psychiatric Association

    Diagnostic and statistical manual of mental disorders: DSM-IV-TR

    (2000)
  • P. Anderson

    Assessment and development of executive function (EF) during childhood

    Child Neuropsychology

    (2002)
  • Lonigan, 2013. Head Toes Knees Shoulders-computerized...
  • C. Blair et al.

    Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten

    Child Development

    (2007)
  • C. Blair et al.

    Multiple aspects of self-regulation uniquely predict mathematics but not letter-word knowledge in the early elementary grades

    Developmental Psychology

    (2015)
  • C. Blair

    School readiness: Integrating cognition and emotion in a neurobiological conceptualization of children's functioning at school entry

    American Psychologist

    (2002)
  • K.C. Brocki et al.

    Interrelations between executive function and symptoms of hyperactivity/impulsivity and inattention in preschoolers: A two year longitudinal study

    Journal of Abnormal Child Psychology

    (2010)
  • S.M. Carlson et al.

    Individual differences in inhibitory control and children's theory of mind

    Child Development

    (2001)
  • S.M. Carlson

    Developmentally sensitive measures of executive function in preschool children

    Developmental Neuropsychology

    (2005)
  • S. Cohen et al.

    Task switching in children

    South carolina bicentennial symposium on attention

    (2001)
  • A. Diamond et al.

    Not quite as grown-up as we like to think: Parallels between cognition in childhood and adulthood

    Psychological Science

    (2005)
  • A. Diamond et al.

    Development of an aspect of executive control: Development of the abilities to remember what I said and to do as I say, not as I do

    Developmental Psychobiology

    (1996)
  • G.J. Duncan et al.

    School readiness and later achievement

    Developmental Psychology

    (2007)
  • Cited by (4)

    • Does preschool children's self-regulation moderate the impacts of instructional activities? Evidence from a randomized intervention study

      2022, Journal of Experimental Child Psychology
      Citation Excerpt :

      Internal consistency reliability for the HTKS task was high (α = .93). Scores on the HTKS task are strongly correlated with other direct measures of EF (e.g., Allan & Lonigan, 2011; Spiegel & Lonigan, 2018). Teacher ratings of self-regulation were obtained using a 44-item hybrid version of the CTRS (Conners, 1999).

    View full text