Figures
Abstract
The quality of staff/child interactions as measured by the Classroom Assessment Scoring System (CLASS) in Early Childhood Education and Care (ECEC) programs is thought to be important for children’s outcomes. The CLASS is made of three domains that assess Emotional Support, Classroom Organization and Instructional Support. It is a relatively new measure that is being used increasingly for research, quality monitoring/accountability and other applied purposes. Our objective was to evaluate the association between the CLASS and child outcomes. Searches of Medline, PsycINFO, ERIC, websites of large datasets and reference sections of all retrieved articles were conducted up to July 3, 2015. Studies that measured association between the CLASS and child outcomes for preschool-aged children who attended ECEC programs were included after screening by two independent reviewers. Searches and data extraction were conducted by two independent reviewers. Thirty-five studies were systematically reviewed of which 19 provided data for meta-analyses. Most studies had moderate to high risk of bias. Of the 14 meta-analyses we conducted, associations between Classroom Organization and Pencil Tapping and between Instructional Support and SSRS Social Skills were significant with pooled correlations of .06 and .09 respectively. All associations were in the expected direction. In the systematic review, significant correlations were reported mainly from one large dataset. Substantial heterogeneity in use of the CLASS, its dimensions, child outcomes and statistical measures was identified. Greater consistency in study methodology is urgently needed. Given the multitude of factors that impact child development it is encouraging that our analyses revealed some, although small, associations between the CLASS and children’s outcomes.
Citation: Perlman M, Falenchuk O, Fletcher B, McMullen E, Beyene J, Shah PS (2016) A Systematic Review and Meta-Analysis of a Measure of Staff/Child Interaction Quality (the Classroom Assessment Scoring System) in Early Childhood Education and Care Settings and Child Outcomes. PLoS ONE 11(12): e0167660. https://doi.org/10.1371/journal.pone.0167660
Editor: Jacobus P. van Wouwe, TNO, NETHERLANDS
Received: August 29, 2016; Accepted: November 17, 2016; Published: December 30, 2016
Copyright: © 2016 Perlman et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: All phases of this study were supported by a grant entitled “What is the impact of early childhood education and care (ECEC) on children’s developmental health" (#234474) from the Canadian Institute for Health Research. Prakesh Shah is supported by an Applied Research Chair in Reproductive and Child Health Services Research from the Canadian Institutes of Health Research (APR-126340). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
For preschool aged children, enrollment in Early Childhood Education and Care (ECEC) in many countries is now the norm [1,2]. Some studies have demonstrated that higher-quality ECEC is associated with improved language development, cognitive functioning, social competence, and emotional adjustment [3–5]. However, others have failed to report such linkages [6–8].
Structural quality indicators (e.g., staff/child ratios, staff education) generally refer to aspects of quality that are more easily quantifiable. As a result, governments tend to focus on regulation of these characteristics. Structural quality is thought to impact children indirectly by setting up the conditions in which children’s direct experiences with the environment take place [9]. For example, when staff-child ratios are better, staff may have more opportunities to have positive interactions with children. Process quality is harder to assess as it focuses on the quality of interactions that children experience directly such as the warmth and responsiveness of the staff that care for children. At least in part due to increasing calls for accountability in education systems, substantial attention is being paid to identifying the kinds of environments that support young children’s learning. Recent studies have focused attention on the quality of interactions children experience and are, therefore, thought to influence children more directly [8,10]. This focus on interactions is driven by some of the most fundamental theories in developmental psychology. These include, but are not limited to, the importance of children’s early social exchanged as outlined by attachment theory [11], Ecological Systems theory’s focus on the child’s interactions with his/her most immediate environment [12] and Vygotsky’s emphasis on learning through social exchanges by supportive “experts” [13]. In keeping with these theories, a key aspect of process quality is the warmth and responsiveness of interactions between staff and children, and the extent to which the interactions scaffold children’s learning and development. The Classroom Assessment Scale (CLASS) is a relatively new measure of the emotional tone and the quality of developmentally appropriate “instruction” (e.g., providing a language/literacy rich environment, opportunities for hands on exploration of natural phenomena, etc.) of ECEC classrooms in a systematic way. In this paper we present a systematic review of the CLASS, which is increasingly influential in the field and child outcomes.
Hamre et al. (2013) [14] present, and provide empirical support for Teaching Through Interactions. This is a conceptual framework that identified three key domains in teacher-child interactions. These domains are Emotional Support, Classroom Organization and Instructional Support. While this work largely discusses instruction within the formal education system it is based on theories that apply equally well to the preschool years. Specifically, in keeping with attachment [11], Emotional Support focuses on staffs’ ability to create warm, safe, responsive and predictable environments that help children because secure, autonomous learners. Classroom Organization captures staffs’ ability to structure the room in a way that minimizes behavioral problems and allows children to focus on their attention on learning. Finally, in keeping with Vygotskian thinking, Instructional Support focuses on the extent to which staff builds on children’s existing knowledge and scaffold leaning. The importance of these domains is supported by a body of research that links them to a host of child outcomes, broadly construed (see Cappella et al. [15] for a comprehensive review of child outcomes associated with these domains of interactions between teachers and children).
The CLASS is a key measure in this important research area as it is specifically designed to capture these three domains. Furthermore, the CLASS has also been central in the development of professional development programs for early childhood educators (e.g., MyTeachingPartner [16]) that are built around this fundamental view of good caregiving/instruction for young children. Thus, the contribution of this systematic review and meta-analysis lies within the current theoretical discussion about the importance of teacher-child interactions as a key driver of teacher effectiveness.
The CLASS is one of several classroom level measures of ECEC process quality. The most frequently used of these is the Early Childhood Environment Rating Scale–Revised (ECERS-R [17]). The ECERS-R is a global measure of quality that includes both structural aspects of the environment (e.g., the physical environment) and process quality (e.g., the kind of language that staff direct to children). There is a substantial body of research linking the ECERS-R to other measures of quality as well as to child outcomes [18–20]. The Observational Record of the Caregiving Environment (ORCE [21]) is another measure but research using the ORCE is largely based on one sample of approximately 1000 participants. As a result, it lacks the evidence base for extensive use or for a systematic review/meta-analysis. While the CLASS is a relatively new measure (certainly compared to the ECERS-R), it is supported by a growing body of evidence and grounded in a substantial theoretical/conceptual framework.
The growing importance of the CLASS is evidenced by the fact that it is now used in statewide ECEC program quality assurance systems in Arizona, Illinois, Maryland, Massachusetts, Minnesota, New Mexico, Oklahoma, Vermont, Virginia and Washington [22]. These assessments are used to make key funding and quality improvement decisions. Some quality assurance programs also make ratings public so that parents can use ratings in selecting placement for their children. Having access to empirically based measures of ECEC quality that are known to be associated with child outcomes is important for a range of stakeholders including parents and the people parents turn to for advice (e.g., their child’s pediatrician), funders, policymakers and others. Despite rapid increases in use of the CLASS in research and in high stakes accountability settings, associations between the CLASS and child outcomes have not been systematically reviewed.
We chose to cast a wide net regarding the child outcomes we included in this review. This reflects an understanding of the classroom context as having an impact on children that goes “beyond achievement tests” [15]. In keeping with this view, we set out to compile studies that used a broad range of outcomes and included information about young children’s very early academic outcomes (including language and early literacy skills); their health/wellbeing and their social-emotional development.
The objective of this review was to evaluate associations between CLASS domain and dimension scores in classrooms that served preschool aged children and children’s concurrent or subsequent social, emotional and cognitive outcomes. A secondary aim was to explore associations between specific CLASS domains and dimensions and specific child outcomes (e.g., Emotional Support to children’s social outcomes, Instructional Support for Learning to language and mathematics or Language Modeling to language outcomes).
Method
Selection of Studies
Searches for potential studies were conducted in three ways. First, the electronic databases, PsycINFO, Medline, and ERIC, were searched for papers that were published from the year each journal began until July 3, 2015. Search terms are provided in Tables A-D in S1 File. Second, websites for datasets that provide key data on ECEC quality and child outcomes were searched. Specifically, websites for the following databases were searched: Cost, Quality and Outcomes Study (CQO) [23]; Early Childhood Longitudinal Study (ECLS) [24]; Effective Provision of Pre-School Education (EPPSE) Project [25]; Head Start Impact Study (HS) [26]; National Center for Early Development and Learning (NCEDL) Multi-State Study of Pre-Kindergarten [27]; State-Wide Early Education Program Study (SWEEP) [27]; Family and Child Experiences Survey (FACES) [28]; and the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care and Youth Development [29]. Finally, reference lists of identified studies were searched to locate additional studies. Searches were limited to English language articles only.
Type of Participants
ECEC programs are generally divided into classrooms that serve children of different ages. We focused on research on classrooms that served preschool aged children (i.e., children approximately between the ages of 30 and 72 months) because much of the research has focused on this group [30,31] as it is the largest group of children cared for in a single type of ECEC setting [32]. The inclusion criteria used in this study and the rationale for each criterion are provided in Table 1.
Study Selection and Data Extraction
The selection of eligible documents was done in two steps that involved pairs of independent raters. First, the titles and abstracts of documents were screened and then full text articles were reviewed to determine eligibility. Discrepancies between raters were resolved through discussion and consensus. Relevant child and family characteristics, study characteristics and reported measures of association were extracted using standardized forms by two reviewers and discrepancies were resolved by consensus.
CLASS Domains and Dimensions
The CLASS consists of three Domains: Emotional Support, Classroom Organization and Instructions Support. Each domain is made up of several dimensions. Specifically, Emotional Support is made up of Negative Climate, Positive Climate, Teacher Sensitivity and Regard for Student Perspective). Classroom Organization is made up of Behavior Management, Productivity and Instructional Learning Formats. Instructional Support is made up of Concept Development, Quality of Feedback, Language Modeling, along with Literacy Focus. Factor analysis of the earlier version of the CLASS yielded two domains: Emotional Climate and Instructional Climate [33]. Emotional Climate includes all of the dimensions of the Emotional Support domain and the Behavior Management dimension that later became part of the Classroom Organization domain. Instructional Climate is a composite of Concept Development and Quality of Feedback. Eligible studies used in this paper operationalized the instrument in various ways (see Table 2). When reported, findings for individual dimensions were gathered. This was done to explore the possibility that associations might be higher between specific dimensions that mapped onto specific outcomes more closely than between the broader domains and specific outcomes (e.g., that language modeling might have a stronger association to children’s language skills than Instructional Support). Including associations with dimensions also increased the comprehensiveness of our review of findings about the CLASS.
Outcomes
In an effort to include the many outcomes that have been associated with high quality “instruction” for children [15] we defined “outcomes” broadly. We included any measure of children’s academic competence (e.g., cognitive, language, mathematics) as well as health/wellbeing and social and emotional functioning.
Meta-Analyses
Studies that could be meta-analyzed were drawn from the pool of studies that were eligible for the systematic review. To be meta-analyzed studies had to use identical child outcome measures and had to use an identical operationalization of the CLASS. We adopted a minimum requirement of three studies to conduct a meta-analysis on a particular child outcome. To increase homogeneity among studies that were meta-analyzed, only studies that ensured children’s exposure to the program were included. To do this, we only included studies in which the authors explicitly stated that children had been in the program for a period of time prior to their assessment. We also included analyses in which children’s pre-scores were used as a covariate, or in which gain scores were used in analyses because such analyses required that children have spent a period of time in the program. In addition, only statistics that accounted for covariates (e.g., child and family characteristics) were combined within a single meta-analysis. Finally, to insure independence of samples, when multiple papers drew from the same dataset, only the study with the largest sample size was selected for inclusion in meta-analyses. Thus, within a given meta-analysis, only one coefficient from each sample was included in any one analysis.
Statistical models with quadratic terms assume non-linear associations between the variables. Given the statistics extracted for most studies only tested for linear relationships (correlation coefficients and linear regression coefficients), associations in models using quadratic terms were excluded and only results examining linear relationships alone were used in the meta-analyses. We used random-effects models for meta-analyses. All meta-analyses were conducted using the software Comprehensive Meta-Analysis Version 3 (See S3 File) [68]. We used the I2 statistic [69] as an indicator of statistical heterogeneity across studies. Low I2 values suggest homogeneity between studies.
Assessment of Risk of Biases Among Included Studies
We planned to assess risk of bias in included studies using Newcastle-Ottawa scale [22,70]. However, initial review revealed that because of the correlational nature of research in this area, the standard questions on this checklist did not enable us to discriminate differences in study quality in our sample of studies. All included studies had moderate to high risk of bias due to issues with sample selection, exposure assessment and confounder adjustment. However, the inclusion criteria we used for the meta-analysis selected for the better quality studies. This means that the studies in the meta-analyses used well-researched, standardized measures, they tended to use covariates in their analyses and to ensure that the children in these studies had at least some exposure to the program before exploring whether program quality was associated with their outcomes. Finally, the majority of the papers we included were published in peer-reviewed journals.
Results
Search Results
Details of the search and study selection are provided in Fig 1. Thirty five studies were included in this review (consisting of 39 samples) [7,8,30,31,34,36,38–63], which included 28 journal articles, 6 reports and 1 book chapter.
Description of Studies
The characteristics of the 35 eligible studies are described Table 2 [7,8,30,31,34,36,38–63]. All of the studies were conducted in the USA. Sixteen studies [7,8,30,31,39–41,43,44,47,48,50,56,58,62,63] came from the NCEDL database and two studies from the FACES 2006 project. Also, two studies used data collected in Ohio and Virginia between 2005 and 2007 [45,59]. Thirty-one studies [7,8,30,34,36,39–53,55–65] were longitudinal and 4 studies [31,38,54,67] were cross-sectional.
Of the independent samples, (i.e., excluding overlapping datasets by retaining only the largest sample size), the total sample size was 15,167 preschool-aged children. Sample sizes ranged from 129 to 3584 children (Median = 398). All of the studies had similar numbers of boys and girls (46% to 62% males). All but one study [58] sampled at-risk children, with 47% to 100% of the children coming from low-income families. Children were primarily Caucasian, Hispanic, or Black.
Operationalization of the CLASS
Studies used the CLASS dimensions, domains or total score. Nineteen studies [34,36,38,42,45,46,49–51,53–55,57,59–61,64,65,67] used the most recent version of the CLASS [72], 5 studies [43,48,52,62,63] used an earlier version [73], which did not include the Language Modeling dimension, and 11 studies [7,8,30,31,39–41,44,47,56,58] used CLASS domains (emotional and/or instructional climate) from the earlier version. Since we did not have data on the comparability of scores when these scales were combined vs. analyzed separately we chose the conservative route and tested the Emotional Climate domain separately. In terms of dimensions available for inclusions in this review, 3 studies [34,36,52] examined Language Modeling, 2 [45,51] examined Behavior Management, 1 study [36] examined Positive Climate and 1 study [52] examined Literacy Focus.
Outcomes
As noted above, outcomes were grouped into cognitive (for which Backward Digit was the most frequently used measure), language (where Peabody Picture Vocabulary Test, PPVT; Woodcock-Johnson Letter-Word Identification, WJ-LWI and the Oral and Written Language Scales, OWLS, are used most often), mathematics (where the Woodcock-Johnson Applied Problems, WJ-AP and Early Childhood Longitudinal Study Birth Cohort Math, ECLS-B math, are used most frequently) and social/emotional outcomes (where the Social Skills Rating System, SSRS, and Teacher Child Relationship Scale, TCRS, are used most often). The 55 different child outcomes identified through this process are listed in S2 File.
Systematic Review
Data for 55 different child outcome measures (see Table 2 where all studies are described and S2 File where all outcomes are listed) were reported in included studies. All of this data (which is presented in Tables A-D, S4 File) was included in the systematic review. To simplify this heterogeneity and help with visual interpretation, child outcomes that were reported in 3 or more samples are reported in Figs 2–7. Of the 35 eligible articles, 29 studies [7,8,30,31,34,36,38–45,47,48,50,51,53,54,56,58,60–65,67] provided data for at least one outcome variable that was also included in at least two other samples. Six studies [46,49,52,55,57,59] are not included in Figs 2–7 because they report only outcomes found in less than three samples.
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. Total Score (Two Factor Solution) = Total Score for Emotional Climate and Instructional Climate; Total Score (9 Scale Version) = Total Score with Language Modeling and Literacy Focus dimension not included. For more details, see Table 2 in this manuscript. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].ANational Center for Early Development and Learning Dataset (NCEDL); BHead Start Family and Children Experiences Survey (FACES 2006 Cohort); MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort); UPreschool Curriculum Evaluation Research (PCER, 1999–2003);XOhio Virginia (2005–2006, 2006–2007).
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson's Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File.bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].ANational Center for Early Development and Learning Dataset (NCEDL); MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].ANational Center for Early Development and Learning Dataset (NCEDL); MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters).fSSRS/SSIS problem behaviour also includes individual scales: internalizing and externalizing for Hestenes et al., 2015 [54].ANational Center for Early Development and Learning Dataset (NCEDL); BHead Start Family and Children Experiences Survey (FACES 2006 Cohort); MHead Start Family and Children Experiences Survey (FACES, 2009 Cohort).
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). ANational Center for Early Development and Learning Dataset (NCEDL).
a Abbreviations: Symbols bolded are significant and positive, symbols bolded and italicized are significant and negative, and symbols in grey are non-significant. Star = Zero Order Pearson’s Correlation, Unfilled circle = Beta, Filled square = Unstandardized Coefficient, Black diamond minus white X = T-Test, Key clover = Partial Correlation, Downward arrow = Effect Size, Filled circle = F-Ratio. aTo improve the readability of this complex table, six papers [46,49,52,55,57,59] that had an outcome that appeared in only one or two samples were omitted from this figure. Several analyses from other papers that had idiosyncratic outcomes were also excluded. For a comprehensive display of all of the data for all of the child outcomes see Tables A-D, S4 File. bThis paper is one of a series of Meta-Analyses and Systematic Reviews assessing the relationship between child care quality and children’s outcomes; therefore, superscript letters below are in reference to various large databases that samples in these papers were drawn from. These letters have been kept consistent across the series of papers for our readers. cSamples within papers are described in more detail in Table 2 in this manuscript. dAcronyms for child outcomes are listed in S5 File. eIdentifying Letters (also referred to as Letter Knowledge, Letter-Naming, Naming Letters). ANational Center for Early Development and Learning Dataset (NCEDL).
Figs 2–7 show relationships between domains and dimensions of the CLASS and outcomes in individual studies. Data is presented for 5 language outcomes, 3 mathematics outcomes and 5 social/emotional outcomes. For language outcomes, a high number of significant associations for the OWLS, PPVT, WJ Letter Word ID (and letter identification measures that were not part of the WJ) and WJ Rhyming were identified in the included studies. However, these results mostly came from studies using NCEDL database. Few significant associations remained when the studies that used the NCEDL [7,8,30,31,39–41,43,44,47,48,50,56,58,62,63] dataset were excluded. Results from studies that used NCEDL data included in this systematic review should be interpreted with caution as the samples in these publications are drawn from a single dataset and therefore overlap. There were a few positive associations between CLASS scores and the math outcome of WJ Applied Problems but none with the ECLS-B Mathematics measure. Few positive significant associations were reported between the various CLASS domains and dimensions and measures of child outcomes.
We also explored whether any one domain or dimension of the CLASS had a stronger pattern of associations with any of our outcomes. We paid particular attention to CLASS domains and dimensions that were conceptually linked to specific child outcomes (e.g., Emotional Support to children’s social outcomes, Instructional Support for Learning to language and mathematics or Language Modeling to language outcomes). Across domains and dimensions there were few associations identified and most of them used from NCEDL data. Overall a consistent lack of relationships between the CLASS and the various outcomes reported in the literature was observed.
We also explored whether accounting for covariates influenced the pattern of results. We did this by comparing results in studies that provided both a Pearson’s correlation and either partial correlations, betas or unstandardized regression coefficients. Five studies [7,38,42,43,63] presented these statistics for the same operationalization of the CLASS and child outcome. Findings across these five studies were highly consistent indicating that when covariates were added there were fewer significant associations compared to the Pearson’s correlations. The one exception to this was Sabol, 2013 [63] in which the significance level of most of the Pearson correlations was not reported. As a final check, we compared studies that used overlapping samples and found that results were highly consistent among them.
Meta-Analyses
When samples overlapped across studies we retained only the study with the largest sample. Thus, 10 studies [7,8,30,31,39,44,50,56,58,62] from the NCEDL database were dropped. We also dropped 1 study [46] with insufficient statistics, 1 study [67] with classroom level analyses, and 4 studies [52,55,59,63] with idiosyncratic outcomes. This left a total of 19 independent samples [34,36,38,40–43,45,47–49,51,53,54,57,60,61,64,65] that provided 28 unique relationships of a particular operationalization of CLASS and a child outcome that meet our criteria for the meta-analyses (See Figs 8–10).
Classroom Organization.
The meta-analysis (Fig 8) revealed significant but small correlations between the Classroom Organization domain and Pencil Tapping (N = 3757, pooled correlation coefficient 0.06; 95% CI: 0.04 to 0.09, I2 = 0%).). The relationships between the Classroom Organization domain and PPVT Vocabulary (N = 3172), WJ LWI (N = 3826), WJ Applied Problems (N = 4976), and SSRS Social Skills (N = 3115) subscales were not significant.
Emotional Support.
The meta-analysis (Fig 9) revealed no significant associations between the Emotional Support domain and Pencil Taping (N = 3757), PPVT Vocabulary (N = 3675), WJ Letter Word ID (N = 1794), WJ Applied Problems (N = 4024), and SSRS Social Skills (N = 2820).
Instructional Support.
The meta-analysis (Fig 10) revealed significant but small correlation between the Instructional Support domain the SSRS Social Skills subscale (N = 3556, 0.09; 95% CI: 0.02 to 0.12, I2 = 3.6%). The relationships between the Instructional Support domain and PPVT (N = 7740) and WJ Letter Word ID (N = 7146) were not significant. The relationship between Instructional Support and the WJ Applied Problems (N = 7531) was also not significant.
Discussion
The CLASS is a relatively new measure of ECEC quality that focuses on constructs that are supported by a long history of theory (e.g., attachment theory, ecological systems theory, etc.) and research on the kids of environments that support child development. The field has embraced the measure as evidenced by its growing use in Quality Ratings and Improvement Systems in the US. One of the appeals of the CLASS is that the measure developers have paid close attention to testing its psychometric properties. Thus, despite being a relatively new measure we found 35 studies that could be used in this review. The quick uptake of this measure in research studies allows us to draw some conclusions about the extent to which it is associated with child outcomes.
In this systematic review and meta-analyses, we found a limited number of significant associations between the CLASS and the language social outcomes available in the literature. This was especially true when the large number of studies that used the NCEDL data was accounted for. Meta-analyses of studies that used the same operationalization of the CLASS, had identical outcome measures and had statistics that could be converted to r revealed some relationships between the different domains of the CLASS and child outcomes. Of the 14 meta-analyses we conducted, 2 showed significant, although small effect sizes. The low I2 values found across these analyses suggest that the studies within each meta-analysis were homogeneous and support our conservative approach of only meta-analyzing studies that were quite similar to one another in terms of operationalization of the CLASS and child outcome.
The lack of relationships is especially surprising in cases in which the CLASS domains maps onto a specific child outcome measure most closely (e.g., Emotional Support and children’s Social Skills measured using the SSRS). However, because the studies we were able to meta-analyze were limited, we did not have an opportunity to test some of these relationships (e.g., we were not able to meta-analyze the relationship between language modeling and children’s vocabulary). Our finding that associations between the CLASS and child outcomes are limited likely reflects the multitude of family and other factors the impact the outcomes we explored in this paper. Below we discuss some of the methodological limitations of the research we covered. We then provide alternative explanations for the weak associations.
The literature in this area is problematic because: 1) it is heterogeneous and 2) many studies have methodological limitations. First, potential for selection bias in the included studies in this review is non-trivial as a number of studies did not randomly select programs and many programs and parents decline to participate. For example, 16 studies [7,8,30,31,39–41,43,44,47,48,50,56,58,62,63] included in this review used data from the NCEDL database. The NCEDL is a large-scale study that draws on randomly selected programs from a number of diverse states. While this study is clearly an important one, it consists of a single sample. Approximately 78% of randomly selected programs in this study agreed to participate while 55–61% of parents from these programs agreed for their children to take part in the study. Thus, even in this relatively methodologically strong study, the final sample may not be representative of the general population of children. Moreover, sample self-selection may contribute to the low variability in CLASS scores observed. With few exceptions, the standard deviations reported for the CLASS domains and dimensions were smaller than 1. This self-selection may result in an over estimate of the quality of existing programs as better programs may be more willing to participate in research on program quality. It may be possible to achieve higher response rates in studies that are part of government initiatives to improve regulation of ECEC programs. Thus, researchers and government or other agencies involved in oversight of ECEC programs need to work together towards improving the quality of research in this area. In general parents who agree to participate in research tend to be better educated. This parent self-selection likely skews the types of children who participate in published studies. Another limitation of the samples used in these CLASS studies is that they all come from the United States. Clearly more research is needed to explore the role of the process quality, as captured in the CLASS, in other jurisdictions.
Second, scores on the domains and dimensions of the CLASS; though consistent across programs, were not always favorable. Scores for Instructional Support were consistently low with values at around “2” whereas Classroom Organization and Emotional Support scores were higher at around “5” on this 7-point scale. This indicates that Instructional Support is an area in particular need of quality improvement across programs. The low scores observed for some domains and dimensions are especially worrisome given our speculation that these are likely to be overestimates of quality due to sample self-section issues.
Finally, there was significant heterogeneity in the outcomes included in these studies. There are two versions of the CLASS, some researchers reported a total score, others created their own profiles from different CLASS codes. To maximize the number of studies we could draw from we attempted to “streamline” the CLASS scales as much as possible by equating domains that were made up of very similar dimensions and including studies that provided data on some dimensions only. There was also substantial variability in the statistics that were reported which further limited our ability to meta-analyze findings across papers. Finally, there is enormous variability in the covariates used in different studies. For example, some studies reported zero order correlations using no covariates [31], while other studies reported only partial correlations or regressions that included anywhere between 5 [39] and 22 [34] covariates controlling for child, family and program factors in their analyses [32]. Clearly studies in this area need to be conducted and reported more consistently so that findings can be integrated more effectively in the future.
Studies about the effects of child care generally have some methodological limitations. For example, for ethical as well as logistical reasons, it is not possible to randomly assign children to programs of differing quality. As a result, all of the studies we included are observational/correlational and causal conclusions about the link between the CLASS and child outcomes cannot be drawn. Reporting of study methodologies and results is inconsistent and at times incomplete. For example, a measure of variability was not reported in a number of studies [7,55,65–67]. However, it is worth noting that the studies included in our meta-analyses used psychometrically strong measurements, had large sample sizes, included many covariates in the analyses in an effort to mitigate the program and parent self-selection issues discussed above, and ensured that children who were assessed had been exposed to the programs. Thus, while the studies we included suffered from substantial limitations, they also had many strengths.
One major drawback of this heterogeneity and the number of studies available for meta-analysis is that we could not test the effects of possible moderators on the relationship between the CLASS and child outcomes. Variables of particular interest are whether the duration of exposure and socioeconomic backgrounds of children influence associations between the CLASS and child outcomes. In general, ECEC programs are expected to make the greatest impact on children from low SES backgrounds. While we were not able to test SES as a moderator, the studies we included generally over represented lower income children. Future reviews based on additional studies to be published in the future in this area, will hopefully be able to test these issues more directly.
Other explanations for the small effect sizes found in this review include the possibility that higher quality programs only impact children who require especially supportive environments. The differential susceptibility hypothesis [74] posits that children who are more vulnerable for genetic, temperamental or other reasons, may be differentially impacted by the extent to which their surroundings are supportive. Thus, overall effects may be eliminated/reduced when aggregating across diverse children within a given sample. Another possible explanation is that the CLASS measures quality at the classroom level, while classrooms in the early childhood sector are staffed by multiple adults. Measures like the Caregiver Interaction Scale [75], which are collected at the staff level, are worth exploring. However, these generally focus only on warmth and sensitivity without the emphasis on developmentally appropriate instruction offered by the CLASS. Discrepancies in the unit of analysis may be contributing to the lack of associations reported in this review. In addition, it is possible that other aspects of interaction should be attended to. For example, Hamre et al. (2013) [14] mention the “use of emotion words and emotion coaching” as “unique elements of classroom interactions not measured by the CLASS” (p. 464). Alternatively, child outcomes might be better predicted by more comprehensive measures such as the ECERS-R described earlier, that capture both structural and process quality. In addition, while we set out to collect a broad range of child outcomes, we could only report on outcomes that were found in the literature. Some outcomes (e.g., gross and fine motor abilities) are notably absent and could explain the lack of associations observed in this study.
Future studies should look at the presence of non-linear relationships between quality characteristics and child outcomes, as there may be some threshold value in ECEC quality that changes the direction of the association. Such relationships cannot be adequately captured by statistical indices that assume linear relationships reported here. The possibility that there are thresholds of quality in terms of impacts on children is gaining attention from researchers. This is illustrated by the recent publication of a special issue on this topic [76]. However, one study that tested this possibility directly with the CLASS did not provide support for the threshold hypothesis for this measure [77]. Finally, one of the strengths of this study is that we cast a very wide net in terms of our initial searches. This resulted in many “hits” that were not, in the end, relevant to the goals of this study. We then developed and systematically applied a set of selection criteria with the aim of having a relatively homogenous sample of studies for subsequent analysis. The downside of this approach is that it was very labour intensive. The upside is that we can be more confident that our searches were comprehensive.
As described earlier, the CLASS is being used increasingly as part of quality assurance programs. It is also used as an integral part of professional development (PD) programs where with the help of a coach, caregivers use the CLASS as a lens through which staff co-view videos of their own interactions with children to improve their practice. The logic of such PD programs stems from the rich literature that formed the basis for approaches like Teaching Through Interaction and informed the development of the CLASS. Yet, given the results of the current systematic review and meta-analysis, clearly more research about what aspects of the classroom environment are associated with child outcomes is needed.
Conclusion
The CLASS operationalizes difficult constructs such as scaffolding and contingent responding by ECEC staff, constructs that are thought to be key in supporting children’s development. The CLASS domain that was most closely linked to child outcomes was Instructional Support. Sadly, it is also, by far, the lowest scoring domain across studies. Very few effects were found for Classroom Organization and Emotional Support. However, it is important to note that based on these findings we cannot conclude that programs of lower or higher quality than what we found in the literature would not impact children. As noted above, it may well be that there are threshold/levels of quality that must be met for quality to impact children. One of the most important conclusions of this study is that more research, using stronger methodologies and a variety of samples is needed. Research with samples from outside the US is especially important. However, despite all of the methodological limitations discussed above, there is a growing body of research about associations between the CLASS and child outcomes and it is important to systematically review the existing literature about this influential measure. Unfortunately, based the existing literature, it appears that associations between the CLASS and child outcomes are quite limited.
Supporting Information
S4 File.
Systematic Review Tables CLASS, Tables A-D.
https://doi.org/10.1371/journal.pone.0167660.s004
(PDF)
Acknowledgments
Special thanks to Charles Pascal, Elaine Baxter-Trahair, Debra Mathews, Christa Japel, Lynn Karoly, Deborah Phillips and Elizabeth Uleryk. We thank Jenaya Webb, OISE/UT’s Public Services Librarian for her invaluable assistance throughout this project. Thank you also to Stephanie Aglipay, Teresa Pietropaolo Anastasia Bukowski and Nadina Mahadeo for all of their help with the huge organizational task associated with this project.
Author Contributions
- Conceptualization: MP OF BF EM JB PSS.
- Data curation: MP OF BF EM JB PSS.
- Formal analysis: MP OF BF EM JB PSS.
- Funding acquisition: MP PSS.
- Investigation: MP OF BF EM JB PSS.
- Methodology: MP OF BF EM JB PSS.
- Project administration: MP.
- Resources: MP.
- Supervision: MP.
- Validation: MP OF BF EM JB PSS.
- Visualization: MP OF BF EM JB PSS.
- Writing – original draft: MP OF BF EM PSS.
- Writing – review & editing: MP OF BF EM JB PSS.
References
- 1.
Laughlin L. Who’s minding the kids? Child care arrangements: spring 2011. Household economic studies. United States: United States Census Bureau; 2013: 1–23. Available from: http://www.census.gov/prod/2013pubs/p70-135.pdf
- 2.
Statistics Canada. Child care: an eight-year profile. The Daily. 5 Apr 2006. Available from: http://www.statcan.gc.ca/daily-quotidien/060405/dq060405a-eng.htm. Cited 1 July 2015.
- 3. Peisner-Feinberg E, Burchinal M, Clifford R, Culkin M, Howes C, Kagan S, et al. The relation of preschool child-care quality to children’s cognitive and social developmental trajectories through second grade. Child Dev. 2001;72: 1534–53. pmid:11699686
- 4. Clarke-Stewart K, Lowe Vandell D, Burchinal M, O’Brien M, McCartney K. Do regulable features of child-care homes affect children’s development? Early Child Res Q. 2002;17: 52–86.
- 5.
Burchinal P, Kainz K, Cai K, Tout K, Zaslow M, Martinez-Beck I, et al. Early care and education quality and child outcomes. Washington, DC, US: Child Trends; 2009. Available from: http://www.acf.hhs.gov/sites/default/files/opre/early_ed_qual.pdf. Cited 13 July 2015.
- 6. Early D, Maxwell K, Burchinal M, Bender R, Ebanks C, Henry G, et al. Teachers’ education, classroom quality, and young children’s academic skills: results from seven studies of preschool programs. Child Dev. 2007;78: 558–80. pmid:17381790
- 7. Howes C, Burchinal M, Pianta R, Bryant D, Early D, Clifford R, et al. Ready to learn? children’s pre-academic achievement in pre-kindergarten programs. Early Child Res Q. 2008;23: 27–50.
- 8. Mashburn A, Pianta R, Hamre B, Downer J, Barbarin O, Bryant D, et al. Measures of classroom quality in prekindergarten and children’s development of academic, language, and social skills. Child Dev. 2008;79: 732–749. pmid:18489424
- 9. NICHD Early Child Care Research Network. Child-care structure→process→outcome: direct and indirect effects of child-care quality on young children’s development. Psychol Sci. 2002;13: 199–206. pmid:12009038
- 10. Burchinal M, Roberts J, Riggins R Jr, Ziesel S, Neebe E, Bryant D. Relating quality of center based child care to early cognitive and language development longitudinally. Child Dev. 2000;71: 339–57. pmid:10834469
- 11.
Bowlby J. Attachment and loss, Vol. 1: Attachment. Vol. 1. New York: Basic Books; 1969.
- 12. Bronfenbrenner U. Ecology of the family as a context for human development: research perspectives. Dev Psychol. 1986;22: 723–42.
- 13. Fernyhough C. Getting Vygotskian about theory of mind: Mediation, dialogue, and the development of social understanding. Dev Rev. 2008;28: 225–62.
- 14. Hamre B, Pianta R, Downer J, DeCoster J, Mashburn A, Jones S, et al. Teaching through interactions: Testing a developmental framework of teacher effectiveness in over 4,000 classrooms. Elem Sch J. 2013;113: 461–87.
- 15.
Cappella E, Aber J, Kim H. Teaching beyond achievement tests: Perspectives from developmental and education science. In Gitomer DH, Bell CA, editors. Handbook of research on teaching. Washington, DC: American Educational Research Association; 2016. pp. 249–347.
- 16. Pianta R, Mashburn A, Downer J, Hamre B, Justice L. Effects of web-mediated professional development resources on teacher–child interactions in pre-kindergarten classrooms. Early Child Res Q. 2008;23: 431–51. pmid:25717217
- 17. NICHD Early Child Care Research Network. Characteristics of infant child care: factors contributing to positive caregiving. Early Child Res Q. 1996;11: 269–306.
- 18.
Henry G, Henderson L, Ponder B, Gordon C, Mashburn A, Rickman D. Report of the findings from the early childhood study: 2001–02. 2003. Georgia State University, School of Policy Studies. Available from: http://eric.ed.gov/?id=ED481261
- 19. McCartney K. Effect of quality of day care environment on children’s language development. Dev Psychol. 1984;20: 244–60.
- 20. Phillips D, McCartney K, Scarr S. Child-care quality and children’s social development. Dev Psychol. 1987;23: 537–43.
- 21. NICHD Early Child Care Research Network. Observational record of the caregiving environment. Early Child Res Q. 1996;11: 269–306.
- 22.
National Center on Child Care Quality Improvement. Use of ERS and other program assessment tools in QRIS. Fairfax, VA: Administration for Children & Families Office of Child Care; 2013; pp. 1–14. Available from: https://occqrisguide.icfwebservices.com/files/QRIS_Program_Assess.pdf. Cited July 13 2015.
- 23.
Cost, Quality and Outcomes Study (CQO). Frank Porter Graham Child Development Institute. Available from: http://fpg.unc.edu/search/apachesolr_search/costqualityoutcome. Cited 8 July 2015.
- 24.
Early Childhood Longitudinal Program (ECLS). National Center for Education Statistics. Available from: http://nces.ed.gov/ecls/. Cited 1 July 2015.
- 25.
Effective Pre-School, Primary and Secondary Education (EPPSE). Institute of Education, University College London. Available from: http://www.ioe.ac.uk/research/153.html. Cited 1 July 2015.
- 26.
Head Start Impact Study (HS). Web site. [Internet]. Administration for children and families. Available from: http://www.acf.hhs.gov/programs/opre/research/project/head-start-impact-study-and-follow-up. Cited 1 July 2015.
- 27.
Early D, Barbarin O, Bryant B, Burchinal M, Chang F, Clifford R, et al. Pre-Kindergarten in eleven states: NCEDL’s multi-state study of pre-kindergarten and study of state-wide early education programs (SWEEP). Chapel Hill, NC: FGP Child Development Center; 2005. Available from: http://fcd-us.org/sites/default/files/Prekindergartenin11States.pdf. Cited 1 July 2015.
- 28.
Head Start Family And Child Experiences Survey (FACES) series. Inter-University Consortium for Political and Social Research. Available from: https://www.icpsr.umich.edu/icpsrweb/ICPSR/series/236. Cited 1 July 2015.
- 29.
Study of Early Child Care and Youth Development (NICHD). U.S. Department of Health and Human Services, National Institutes of Health. 2015. Available from: http://www.nichd.nih.gov/research/supported/seccyd/Pages/datasets.aspx. Cited 8 July 2015.
- 30. Early D, Bryant D, Pianta R, Clifford R, Burchinal M, Ritchie S, et al. Are teachers’ education, major, and credentials related to classroom quality and children’s academic gains in pre-kindergarten? Early Child Res Q. 2006;21: 174–95.
- 31. Burchinal M, Howes C, Pianta R, Bryant D, Early D, Clifford R, et al. Predicting child outcomes at the end of kindergarten from the quality of pre-kindergarten teacher–child interactions and instruction. Appl Dev Sci. 2008;12: 140–53.
- 32. NICHD Early Child Care Research Network. Child outcomes when child care center classes meet recommended standards for quality. Am J Public Health. 1999;89: 1072–7. pmid:10394318
- 33. La Paro K, Pianta R, Stuhlman M. The classroom assessment scoring system: findings from the prekindergarten year. Elem Sch J. 2004;104: 409–26.
- 34.
Aikens N, Tarullo L, Hulsey L, Ross C, West J, Xue Y. ACF-ORPRE report: a year in head start: children, families and programs]. Washington DC: Administration for Children and Families; 2010 pp. 1–36. Available from: http://files.eric.ed.gov/fulltext/ED517213.pdf. Cited 13 July 2015.
- 35.
Hulsey L, Aikens N, Xue Y, Tarullo L, West J. ACF-OPRE Report: Data tables for FACES 2006: a year in head start report. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation; 2010. Available from: http://www.acf.hhs.gov/sites/default/files/opre/year_data_tables.pdf
- 36.
Aikens N, Moiduddin E, Xue Y, Tarullo L, West J. Data tables for child outcomes and classroom quality in FACES 2009 report. OPRE report 2012- 37b. Washington, DC: Administration for Children and Families. 2012; pp. 1–200. Available from: http://www.acf.hhs.gov/sites/default/files/opre/data_tables_for_child_outcomes_and_classroom_quality_in_faces_2009.pdf. Cited 13 July 2015.
- 37.
Moiduddin E, Aikens N, Tarullo L, West J, Xue Y. Child outcomes and classroom quality in FACES 2009. OPRE report 2012-37a. Washington DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. 2012; pp. 1–46. Available from: http://files.eric.ed.gov/fulltext/ED539265.pdf. Cited 13 July 2015.
- 38. Bulotsky-Shearer R, Bell E, Carter T, Dietrich S. Peer play interactions and learning for low-income preschool children: the moderating role of classroom quality. Early Educ Dev. 2014;25: 815–40.
- 39. Burchinal M, Vandergrift N, Pianta R, Mashburn A. Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs. Early Child Res Q. 2010;25: 166–76.
- 40.
Burchinal M, Kainz K, Cai K. How well do our measures of quality predict child outcomes? A meta-analysis and coordinated analysis of data from large-scale studies of early childhood settings. In: Zaslow M, Martinez-Beck I, Tout K, Halle T, editors. Quality measurement in early childhood settings. Baltimore, MD: Paul H Brookes Publishing; 2011. pp. 11–31. Available from: http://www.researchconnections.org/childcare/resources/21062
- 41. Burchinal M, Field S, López M, Howes C, Pianta R. Instruction in Spanish in pre-kindergarten classrooms and child outcomes for English language learners. Early Child Res Q. 2012;27: 188–97.
- 42. Burchinal M, Vernon-Feagans L, Vitiello V, Greenberg M, The Family Life Project Key Investigators. Thresholds in the association between child care quality and child outcomes in rural preschool children. Early Child Res Q. 2014;29: 41–51.
- 43. Curby T, Brock L, Hamre B. Teachers’ emotional support consistency predicts children’s achievement gains and social skills. Early Educ Dev. 2013;24: 292–309.
- 44.
Dang T, Farkas G, Burchinal M, Duncan G, Vandell D, Li W, et al. Preschool center quality and school Readiness: quality main effects and variation by demographic and child characteristics. Society for Research on Educational Effectiveness; 2011. Available from: http://eric.ed.gov/?id=ED519004. Cited 13 July 2015.
- 45. Dobbs-Oates J, Kaderavek J, Guo Y, Justice L. Effective behavior management in preschool classrooms and children’s task orientation: enhancing emergent literacy and language development. Early Child Res Q. 2011;26: 420–9.
- 46. Dominguez X, Vitiello V, Maier M, Greenfield D. A longitudinal examination of young children’s learning behavior: child-level and classroom-level predictors of change throughout the preschool year. Sch Psychol Rev. 2010;39: 29–47.
- 47. Dotterer A, Burchinal M, Bryant D, Early D, Pianta R. Universal and targeted pre-kindergarten programmes: a comparison of classroom characteristics and child outcomes. Early Child Dev Care. 2012;183: 931–50.
- 48. Downer J, López M, Grimm K, Hamagami A, Pianta R, Howes C. Observations of teacher–child interactions in classrooms serving Latinos and dual language learners: applicability of the classroom assessment scoring system in diverse settings. Early Child Res Q. 2012;27: 21–32.
- 49. Gosse C, McGinty A, Mashburn A, Hoffman L, Pianta R. The role of relational and instructional classroom supports in the language development of at-risk preschoolers. Early Educ Dev. 2014;25: 110–33.
- 50. Guo Y, Piasta S, Justice L, Kaderavek J. Relations among preschool teachers’ self-efficacy, classroom quality, and children’s language and literacy gains. Teach Teach Educ. 2010;26: 1094–103.
- 51. Guo Y, Tompkins V, Justice L, Petscher Y. Classroom age composition and vocabulary development among at-risk preschoolers. Early Educ Dev. 2014;25: 1016–34. pmid:27660399
- 52. Hamre B, Justice L, Pianta R, Kilday C, Sweeney B, Downer J, et al. Implementation fidelity of MyTeachingPartner literacy and language activities: association with preschoolers’ language and literacy growth. Early Child Res Q. 2010;25: 329–47.
- 53. Hamre B, Hatfield B, Pianta R, Jamil F. Evidence for general and domain-specific elements of teacher-child interactions: associations with preschool children’s development. Child Dev. 2014;85: 1257–74. pmid:24255933
- 54. Hestenes L, Kintner-Duffy V, Wang Y, La Paro K, Mims S, Crosby D, et al. Comparisons among quality measures in child care settings: understanding the use of multiple measures in North Carolina’s QRIS and their links to social-emotional development in preschool children. Early Child Res Q. 2015;30(Part B):199–214.
- 55. Johnson S, Seidenfeld A, Izard C, Kobak R. Can classroom emotional support enhance prosocial development among children with depressed caregivers? Early Child Res Q. 2013;28: 282–90.
- 56. Keys T, Farkas G, Burchinal M, Duncan G, Vandell D, Li W, et al. Preschool center quality and school readiness: quality effects and variation by demographic and child characteristics. Child Dev. 2013;84: 1171–90. pmid:23331043
- 57. Logan J, Piasta S, Justice L, Schatschneider C, Petrill S. Children’s attendance rates and quality of teacher-child interactions in at-risk preschool classrooms: contribution to children’s expressive language growth. Child Youth Care Forum. 2011;40: 457–77.
- 58. Mashburn A, Justice L, Downer J, Pianta R. Peer effects on children’s language achievement during pre-kindergarten. Child Dev. 2009;80: 686–702. pmid:19489897
- 59. McGinty A, Justice L, Piasta S, Kaderavek J, Fan X. Does context matter? explicit print instruction during reading varies in its influence by child and classroom factors. Early Child Res Q. 2012;27: 77–89.
- 60.
Peisner-Feinberg E, Schaaf J. Evaluation of the North Carolina More at Four Pre-Kindergarten Program: Performance and Progress in the Seventh Year (2007–2008) [Internet]. Chapel Hill, NC: FPG Child Development Institute; 2008. Available from: http://fpg.unc.edu/sites/fpg.unc.edu/files/resources/reports-and-policy-briefs/MAF_Yr7_full_report.pdf. Cited 13 July 2015.
- 61.
Peisner-Feinberg E, Schaaf J, LaForett D. Children’s growth and classroom experiences in Georgia’s pre‐k program: findings from the 2011–2012 evaluation study. Chapel Hill, NC: FPG Child Development Institute; 2013. Available from: http://files.eric.ed.gov/fulltext/ED541933.pdf
- 62. Reid J, Ready D. High-quality preschool: the relationship between the socioeconomic composition of preschool classrooms and children’s learning. Early Educ Dev. 2013;24: 1082–111.
- 63. Sabol T, Soliday Hong S, Pianta R, Burchinal M. Can rating pre-k programs predict children’s learning? Science. 2013;341: 845–6. pmid:23970684
- 64. Weiland C, Ulvestad K, Sachs J, Yoshikawa H. Associations between classroom quality and children’s vocabulary and executive function skills in an urban public prekindergarten program. Early Child Res Q. 2013;28: 199–209.
- 65.
West J, Malone L, Hulsey L, Aikens N, Tarullo L. ACF-OPRE report: Head Start children go to kindergarten. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation; 2010. Available from: http://www.acf.hhs.gov/sites/default/files/opre/hs_kindergarten.pdf
- 66.
Malone L, Hulsey L, Aikens N, West J, Tarullo L. ACF-OPRE report data tables for FACES 2006: head start children go to kindergarten report. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation. 2010; pp. 1–164. Available from: http://www.acf.hhs.gov/sites/default/files/opre/hs_kinder_data.pdf. Cited 13 July 2015.
- 67. Xu Y, Chin C, Reed E, Hutchinson C. The effects of a comprehensive early literacy project on preschoolers’ language and literacy skills. Early Child Educ J. 2014;42: 295–304.
- 68.
Comprehensive meta-analysis. Version 3.0. Engelwood, NJ: Biostat; 2005.
- 69. Higgins J, Thompson S, Deeks J, Altman D. Measuring inconsistency in meta-analyses. Br Med J. 2003;327: 557–60.
- 70.
Wells G, Shea B, O’Connel D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa scale (NOS) for assessing the quality if nonrandomized studies in meta-analyses. The Ottawa Hospital Research Institute. Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Cited 1 July 2015.
- 71. Moher D, Liberati A, Tetzlaff J, Altman D, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6.
- 72.
Pianta R, La Paro K, Hamre B. Classroom Assessment Scoring System (CLASS) Pre-k Version. Baltimore, MD: Brookes Publishing; 2007.
- 73.
Pianta R, La Paro K, Hamre B. Classroom Assessment Scoring System [CLASS] Manual: Pre-K. Vol. 23. Baltimore, MD: Brookes Publishing; 2004.
- 74. Ellis B, Boyce W, Belsky J, Bakermans-Kranenburg M, Van Ijzendoorn M. Differential susceptibility to the environment: An evolutionary–neurodevelopmental theory. Dev Psychopathol. 2011;23: 7–28. pmid:21262036
- 75. Arnett J. Caregivers in day-care centers: Does training matter? J Appl Dev Psychol. 1989;10: 541–52.
- 76. Burchinal J, Zaslow M, Tarullo L. Quality thresholds, features, and doses in early care and education: Secondary data analyses of child outcomes. Monogr Soc Res Child Dev. 2016;81: 1–128.
- 77. Hatfield B, Burchinal M, Pianata R, Sideris J. Thresholds in the association between quality of teacher–child interactions and preschool children’s school readiness skills. Early Child Res Q. 2016;36: 561–71.