A head-to-toes approach to computerized testing of executive functioning in young children
Introduction
Self-regulation is an important developmental capacity related to other significant developmental outcomes, including academic and social difficulties and problem behaviors (e.g., Blair, 2002; Eisenberg et al., 2000; Lonigan, Allan, & Phillips, 2017; Lonigan, Spiegel et al., 2017; Vitaro, Brendgen, Larose, & Trembaly, 2005). The construct of self-regulation is broadly defined as consisting of multiple aspects of children’s behavior, including planning, directing, and controlling abilities (e.g., Mahone & Hoffman, 2007). More narrowly, self-regulation can be defined as the partially separable constructs of executive function (EF), attentional control, and effortful control (Allan, Allan, Lerner, Farrington, & Lonigan, 2015; Blair, Urasche, Greenberg, Vernon-Feagans, & Family Life Project Investigators, 2015). From a cognitive perspective, the construct most frequently associated with self-regulation is EF; EF is a multidimensional construct defined by domain-general cognitive processes that are associated with the regulation of emotions and goal-directed behaviors. Specifically, EF is hypothesized to consist of three distinct components: inhibitory control (IC), working memory (WM), and shifting (SH; Miyake, Friedman, Emerson, Witzki, & Howerter, 2000).
EF skills, including IC and WM, are associated with academic outcomes in preschool and early elementary school (e.g., Allan & Lonigan, 2011; Blair & Razza, 2007; Lonigan, Allan et al., 2017; McClelland et al., 2007). With young children, these EF skills have been shown to be longitudinally predictive of academic abilities independent of initial cognitive (e.g., McClelland, Morrison, & Holmes, 2000) and academic skills (e.g., Howse, Lange, Farran, & Boyles, 2003; McClelland et al., 2000; Ponitz, McClelland, Matthews, & Morrison, 2009). Additionally, children with poor EF are at an increased risk of academic difficulties and school dropout in late childhood and early adolescence (Duncan et al., 2007; Eisenberg et al., 2000; Vitaro et al., 2005). Results of meta-analytic studies examining the relations between EF and externalizing behavior disorders, demonstrate moderate associations between self-regulation and symptoms of Attention-Deficit/Hyperactivity Disorder (ADHD; Alderson, Rapport, & Kofler, 2007; Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005) and somewhat inconsistent relations with symptoms of Oppositional Defiant Disorder (ODD) and Conduct Disorder (Pennington & Ozonoff, 1996). Among non-clinical populations of young children, EF has also been shown to be associated with individual differences in externalizing behaviors (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017; Hughes & Ensor, 2011; Rezazadeh, Wilding, & Cornish, 2011; Sulik, Blair, Mills-Koonce, Berry, Greenberg, & The Family Life Project Investigators, 2015). Given the substantial associations between EF and academic and behavioral functioning in young children, the accurate assessment of EF skills may be particularly important for the identification of children at risk for difficulties upon school entry.
Despite the fact that EF skills have consistently been found to relate to a wide array of important developmental outcomes (e.g., McClelland et al., 2000; Willcutt et al., 2005), only minor associations have been found between EF skills and children’s ability to delay rewards (e.g., Allan & Lonigan, 2011). Direct comparisons of delay aversion and IC tasks among children with ADHD indicate that these constructs represent distinct influences on child psychopathology (see Sonuga-Barke, 2002, for a review). Additionally, the nonsignificant negative correlations between delay aversion and IC tasks reported in previous studies of community-based samples of children (e.g., Allan & Lonigan, 2011) offer further support that delay aversion tasks measure a different underlying construct than IC.
Young children’s self-regulation and EF skills are assessed via either report- or performance-based measures. Report-based measures involve self-, parent-, or teacher-ratings of EF skills based on observations of behaviors within everyday situations. Report-based measures of EF offer an easy to administer, time efficient, and ecologically valid alternative to performance-based measures. However, such measures have been criticized as assessing skills only tangentially related to executive functioning (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017; Spiegel, Lonigan, & Phillips, 2017), and, although they are designed to assess the same underlying construct as performance-based measures, the two types of measures often demonstrate only small to moderate associations (Toplak, West, & Stanovich, 2013).
Performance-based measures offer a direct assessment of EF skills. One such measure is the Head-to-Toes task (HTT; Ponitz et al., 2008). The HTT task is an adapted version of the Head-to-Feet assessment (McCabe, Rebello-Britto, Hernandez, & Brooks-Gunn, 2004). The HTT task is a response-conflict EF task (i.e., a task that requires the child to perform a subordinate response while refraining from performing a dominant response) that is played as a game in which participants are told to do the opposite of a command. For instance, participants are told that when they hear the command, “touch your toes” they should touch their head. A more complex version of the HTT task, the Head-Toes-Knees-Shoulders task (HTKS; Ponitz et al., 2009) adds two additional commands. Specifically, participants are told to touch their knees when they hear the command “touch your shoulders” and touch their shoulders when they hear the command “touch your knees.” The head and toes commands are interspersed for the first 10 trials and all four commands are interspersed for the second 10 trials. Although the HTT and HTKS tasks are primarily measures of IC, other self-regulatory skills, such as WM and attention, may also be measured (Ponitz et al., 2008).
The HTKS task is easy to administer and requires few materials and a minimal time commitment. In addition, the HTKS task is designed to be appealing to young children, incorporating aspects of commonly played games with which children are likely to be familiar. Overall, the HTT and HTKS tasks are ideal measures for administration in both the laboratory and field settings and promote task engagement when utilized with young children. Both HTT and HTKS tasks demonstrate strong construct validity, with moderate relations reported between scores on these tasks and performance- and report-based measures of self-regulation (e.g., Allan & Lonigan, 2011; Graziano et al., 2015). Specifically, children with higher performance on the HTKS were rated as having higher regulatory skills by parents (Graziano et al., 2015) and teachers (Ponitz et al., 2009) and as demonstrating stronger EF abilities on other performance-based measures of IC (e.g., Allan & Lonigan, 2011; Fuhs, Farran, & Nesbitt, 2015) and WM (e.g., Graziano et al., 2015; Lonigan, Lerner, Goodrich, Farrington, & Allan, 2016). Consistent with the broader literature on EF, results of studies demonstrate significant concurrent and longitudinal associations between scores on the HTT and HTKS tasks and academic and behavioral outcomes when used with preschool and kindergarten children. Significant positive associations are consistently reported between scores on the HTKS task and measures of literacy (e.g., Allan & Lonigan, 2011; Graziano et al., 2015; Matthews, Ponitz, & Morrison, 2009; McClelland et al., 2007) and mathematics (e.g., Ponitz et al., 2009; Skibbe, Connor, Morrison, & Jewkes, 2011, Wanless et al., 2011) both concurrently and longitudinally. Substantial associations have also been found between the HTKS task and report-based measures of externalizing behavior problems; however, results seem to be discrepant based on the specific symptoms examined (Lonigan, Allan et al., 2017; Lonigan, Spiegel et al., 2017).
Although the HTT and HTKS tasks are useful measures of EF for preschool and kindergarten children, the age range within which they can be reliably utilized may be limited. Results of studies that have used the HTT or HTKS tasks indicate that by kindergarten many children are able to perform these tasks without errors (Ponitz et al., 2008), resulting in ceiling effects and non-normally distributed scores. Results indicate that the HTT and HTKS tasks are most useful and appropriate when used with children prior to kindergarten (Ponitz et al., 2009). Ceiling effects may be especially pronounced in girls, with studies consistently reporting significantly less variability in HTKS scores for girls than for boys (Matthews et al., 2009; Wanless et al., 2013).
An important question about children’s EF concerns the rate and timing of development of EF capacity. Important developments in EF skills occur throughout the first nine years of life, with marked changes occurring between the ages of 3 and 6 (e.g., Diamond & Taylor, 1996; Espy, Kaufmann, McDiarmid, & Glisky, 1999) and between ages 7 and 9 (Anderson, 2002). Optimally, measures of EF would span the age range during which there is rapid growth of EF capacity; however, ceiling effects on the HTT and HTKS tasks (e.g., Matthews et al., 2009) limit the ability to use these measures longitudinally. Moreover, whereas younger children are likely to find the HTT and HTKS tasks fun and game like, it is possible that such tasks are less developmentally appropriate for older children, potentially resulting in suboptimal performance when older children are unmotivated to perform well on a task that they find age-inappropriate. Such distributional problems and motivational issues limit the ability to use these tasks to study the development and developmental significance of EF skills beyond kindergarten.
As a means of addressing the potential problems associated with the HTKS task, Lonigan (2013) developed a computerized version of the HTKS task. Previous research on other neuropsychological performance tasks has similarly addressed distributional and motivational concerns by creating adapted computerized versions of tasks to more accurately detect performance errors (i.e., incorrect or self-corrected responses) by standardizing test administration and providing automated immediate response recording (Luciana, 2003). To the best of our knowledge, no study to date has examined task motivation of children on computerized versus standard task administration; however, findings from studies with adults indicate a strong preference towards computerized task administration (Noyes & Garland, 2008). In the computerized version of the HTKS, participants are presented with an outline of a human body. Just as with the original task, the computerized version requires participants to remember two commands for the first 10 trials and four commands for the second 10 trials. Like the original HTKS task, the computerized version is a response-conflict task, meaning that throughout the task participants are required to do the opposite of each command.
Although there are numerous advantages to using the computerized version of the HTKS task in older children, younger children may have difficulties with the computerized version because computerizing task administration may increase task demands for reasons that are independent of EF skills (e.g., variable knowledge of computers and fine motor skills needed to respond with a mouse; Saçkes, Trundle, & Bell, 2011). Given the strength of results to date for the original HTT and HTKS tasks with younger children, it is important to demonstrate that both the original and computerized versions assess the same underlying construct to the same degree within this age group. Before the longitudinal applications of the computerized HTKS can be examined it is important to first examine the utility of the measure with the children it was originally designed for use with. Therefore, the primary goal of this study was to evaluate the degree to which the original and computerized versions of the HTKS task provided an equivalent measurement of EF for young children. It was hypothesized that the original and computerized versions of the HTKS task would provide equivalent measures of EF in preschool-age children. Children were administered a battery of commonly used performance-based EF, delay, and academic ability tasks, and parents and teachers completed measures of externalizing behaviors. Relations between the computerized HTKS task and these measures were hypothesized to be similar to those previously reported between the HTT and HTKS tasks and similar measures of important developmental outcomes.
Section snippets
Participants
Data from 130 preschool children were collected for this study. Because young children may not be familiar with computer tasks or may have less well-developed fine motor skills that could influence the degree to which their recorded responses reflect the underlying construct of IC, a mouse-tracking task was administered to assess participants’ abilities to manipulate and point with a mouse. Four children were excluded because they were unable to demonstrate competency using a computer mouse or
Results
Descriptive statistics for all measures are shown in Table 1. The majority of children’s scores on the original HTKS fell above the halfway score of 20 out of 40 points (i.e., 60.8%), with 36% of children scoring at or above 30. However, despite the majority of children’s scores falling above the halfway score, examination of the skewness and kurtosis indicated a normal distribution or scores, consistent with findings from Ponitz et al. (2009). Scores on the computerized HTKS task were also
Discussion
The purpose of this study was to examine whether the HTKS-c measured the same underlying construct as the original HTKS task by examining the convergent and divergent validity of these tasks with measures of important developmental outcomes. Overall, the computerized version demonstrated good construct validity when compared to the original HTKS task and other commonly used measures of IC. Further, the HTKS-c demonstrated good utility and convergent validity as evidenced by substantial
Conclusions
Previous studies have examined the psychometric properties and utility of the original HTKS task as a measure of IC in young children (e.g., Allan & Lonigan 2011; Graziano et al., 2015; Ponitz et al., 2009). However, the original version of the task has a number of limitations, including ceiling effects toward the end of kindergarten, motivational issues with older children, and the possibility of errors in administration and scoring during live administration. The computerized version of the
Acknowledgments
This research and report was supported by grants from the Institute of Education Sciences, US Department of Education (R305B090021) and the Eunice Kennedy Schriver National Institute of Child Health and Human Development (HD052120). The views expressed herein are those of the authors and have not been reviewed or approved by the granting agency.
References (62)
- et al.
Identifying unique components of preschool children’s self-regulatory skills using executive function tasks and continuous performance tests
Early Childhood Research Quarterly
(2015) - et al.
The contributions of ‘hot’ and ‘cool’ executive function to children's academic achievement, learning-related behaviors, and engagement in kindergarten
Early Childhood Research Quarterly
(2009) - et al.
Ten-year review of rating scales. V: Scales assessing attention-deficit/hyperactivity disorder
Journal of the American Academy of Child & Adolescent Psychiatry
(2003) - et al.
Executive functioning in preschool children: Performance on A-not-B and other delayed response format tasks
Brain and Cognition
(1999) - et al.
Individual differences in growth in executive function across the transition to school predict externalizing and internalizing behaviors and self-perceived academic success at 6 years of age
Journal of Experimental Child Psychology
(2011) - et al.
Executive function of Spanish-speaking language-minority preschoolers: Structure and relations with early literacy skills and behavioral outcomes
Journal of Experimental Child Psychology
(2016) - et al.
Children at risk for early academic problems: The role of learning-related social skills
Early Childhood Research Quarterly
(2000) - et al.
The unity and diversity of executive functions and their contributions to complex frontal lobe tasks: A latent variable analysis
Cognitive Psychology
(2000) - et al.
Young children’s computer skills development from kindergarten to third grade
Computers & Education
(2011) - et al.
Multi-method assessment of ADHD characteristics in preschool children: Relations between measures
Early Childhood Research Quarterly
(2012)
Schooling effects on preschoolers’ self-regulation, early literacy, and language growth
Early Childhood Research Quarterly
Preliminary construct and concurrent validity of the preschool self-regulation assessment (PSRA) for field-based research
Early Childhood Research Quarterly
Psychological heterogeneity in AD/HD—A dual pathway model of behaviour and cognition
Behavioural Brain Research
Gender differences in behavioral regulation in four societies: The United States, Taiwan, South Korea, and China
Early Childhood Research Quarterly
Validity of the executive function theory of Attention-Deficit/Hyperactivity Disorder: A meta-analytic review
Biological Psychiatry
Attention-Deficit/Hyperactivity Disorder and behavioral inhibition: A meta-analytic review of the stop-signal paradigm
Journal of Abnormal Child Psychology
Examining the dimensionality of effortful control in preschool children and its relation to academic and socioemotional indicators
Developmental Psychology
Relations between inhibitory control and the development of academic skills in preschool and kindergarten: A meta-analysis
Developmental Psychology
Diagnostic and statistical manual of mental disorders: DSM-IV-TR
Assessment and development of executive function (EF) during childhood
Child Neuropsychology
Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten
Child Development
Multiple aspects of self-regulation uniquely predict mathematics but not letter-word knowledge in the early elementary grades
Developmental Psychology
School readiness: Integrating cognition and emotion in a neurobiological conceptualization of children's functioning at school entry
American Psychologist
Interrelations between executive function and symptoms of hyperactivity/impulsivity and inattention in preschoolers: A two year longitudinal study
Journal of Abnormal Child Psychology
Individual differences in inhibitory control and children's theory of mind
Child Development
Developmentally sensitive measures of executive function in preschool children
Developmental Neuropsychology
Task switching in children
South carolina bicentennial symposium on attention
Not quite as grown-up as we like to think: Parallels between cognition in childhood and adulthood
Psychological Science
Development of an aspect of executive control: Development of the abilities to remember what I said and to do as I say, not as I do
Developmental Psychobiology
School readiness and later achievement
Developmental Psychology
Cited by (4)
Does preschool children's self-regulation moderate the impacts of instructional activities? Evidence from a randomized intervention study
2022, Journal of Experimental Child PsychologyCitation Excerpt :Internal consistency reliability for the HTKS task was high (α = .93). Scores on the HTKS task are strongly correlated with other direct measures of EF (e.g., Allan & Lonigan, 2011; Spiegel & Lonigan, 2018). Teacher ratings of self-regulation were obtained using a 44-item hybrid version of the CTRS (Conners, 1999).
Head-Toes-Knees-Shoulders task and EF in two samples of adolescents in Brazil and United States
2023, Frontiers in PsychologyTowards Scalable, Integrative Assessment of Children’s Self-Regulatory Capabilities: New Applications of Digital Technology
2019, Clinical Child and Family Psychology Review