Unpacking intervention effects: Teacher responsiveness as a mediator of perceived intervention quality and change in teaching practice
Section snippets
Professional development to improve early childhood teacher–child interactions
Recent work suggests that PD directly targeting improvements in preschool teacher-child emotional, organizational, and instructional interactions can be effective in improving these behaviors (Bierman et al., 2008; Clements & Sarama, 2007; Domitrovich et al., 2009, Hsieh et al., 2009, Pianta La Paro et al., 2008, Pianta Mashburn et al., 2008, Raver et al., 2008). Most of these empirically supported PD approaches provide some combination of curriculum training and classroom-based coaching to
Unpacking what happens within interventions
Recent reviews of intervention evaluation studies note the progress made in conducting field-based randomized trials as a primary mechanism for determining impact, but also call for going beyond understanding if an intervention works to the unpacking of “how” and why” it works (Berkel, Mauricio, Schoenfelder, & Sandler, 2011; Dane & Schneider, 1998; Domitrovich & Greenberg, 2000; Durlack & DuPre, 2008; Irwin & Supplee, 2012). Some of these questions pertain to implementation. Increased
Intervention implementation fidelity and the NCRECE interventions
As outlined by Fixsen, Naoom, Blase, Friedman, and Wallace (2005) in their synthesis of the implementation research literature, intervention implementation should ideally pay attention to multiple facets of fidelity. Most implementation research has focused specifically on fidelity to the model, which typically involves verifying that key intervention components were delivered as intended (Durlak & DuPre, 2008). Fidelity to the model tends to be assessed in a straightforward manner, often with
New directions in intervention implementation research: the role of perceived quality of delivery and participant responsiveness
Although much previous implementation research has focused on fidelity, a growing interest in the quality of delivery is emerging as an understudied aspect of implementation (Berkel et al., 2011; Durlak & DuPre, 2008). Intervention quality of delivery comprises an assessment of whether the core intervention components were delivered in a clear, comprehensible, and enthusiastic manner (Berkel et al., 2011). Evidence suggests that participants' perception of intervention quality may provide a
Current study
Using Berkel et al. (2011) model as the frame, the current study examines the associations among early childhood teachers' perceived quality of delivery, their responsiveness to the intervention across several indicators, and subsequent improved instructional interactions. More specifically, the research questions for this study are: (1) To what extent does teachers' perception of the intervention quality of delivery relate to several aspects of their responsiveness (attendance, active
NCRECE overview and participants
The NCRECE Professional Development Study was a randomized, controlled evaluation of two forms of PD support designed to improve preschool teachers' interactions with children. The study involved a design that included two phases, chronologically ordered. In the first study phase, teachers either were randomly assigned to take the NCRECE course delivered over 14 weeks or were in a control group. Then, teachers were re-randomized and either participated in MyTeachingPartner coaching (Pianta La
Perceived intervention quality of delivery
Teachers were asked to report their perceptions of the course instructor and coach quality of delivery. Items came from previous work that found the scale related to participation in a coaching intervention (Downer, LoCasale-Crouch, Hamre, & Pianta, 2009) and were slightly reworded for this study so they could be used in the course intervention. Scale items were completed outside of the course/coaching on a survey provided by the researchers. Following completion of the course, participants
Engagement
Instructors and coaches completed surveys at the end of the intervention regarding the extent to which teachers were engaged during the intervention. Items came from previous work examining teacher engagement within MTP coaching (Downer et al., 2009) and were slightly reworded for this study so they could be used in the course intervention as well. Instructors responded to 10 items on a 5-point scale (Cronbach's alpha = .96). Items included, “This teacher brought his/her own ideas and questions
Instructional interactions
The effectiveness of teacher-child interactions was rated using the Classroom Assessment Scoring System (CLASS; Pianta La Paro et al., 2008, Pianta Mashburn et al., 2008). The CLASS is an observational assessment of global teacher-child interactions in classroom settings. Each dimension is rated on a 7-point scale with behavioral indicators and anchor point descriptions provided for low, medium, and high levels of that dimension. For the purposes of this study, we examined the CLASS
Results
Our primary goal was to determine whether teachers' perceptions of the intervention quality of delivery were related to their responsiveness to the intervention, and whether responsiveness, in turn, was related to changes in their instructional support (see Fig. 1). Given our interest in testing responsiveness as an intervening variable in the relation between quality and changes in instructional support, we tested our model using mediation analysis (Baron & Kenny, 1986). We examined separate
Course
The bivariate correlations between the variables included in the course phase mediation model are presented in Table 2. These correlations suggest that perception of quality of delivery is significantly related to all three of the responsiveness measures, but neither quality of delivery nor the measures of responsiveness have a significant bivariate relation with posttest instructional support scores. Further, given the low to moderate correlations among the different measures of
Coaching
The bivariate correlations between the variables included in the coaching phase mediation model are presented in Table 4. These correlations suggest that quality is marginally related to coach-rated engagement (p = .06) and significantly related to teacher attendance (i.e., number of cycles and time spent using the web). Rated engagement and the number of cycles (attendance) were significantly related to posttest instructional support scores, but the time spent using web resources (home practice)
Discussion
The present study aimed to use a theoretical framework not previously tested (Berkel et al., 2011) to increase understanding of the ways in which implementation of two interventions (the NCRECE course and MTP coaching) already demonstrated in intent-to-treat analyses to improve the quality of teacher-child interactions (Downer et al., 2012, Hamre et al., 2012) might work. In both interventions explored, responsiveness related to change in teachers' instructional interactions and teachers'
Acknowledgements
The authors wish to thank the generous programs and teachers who participated in this study. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A060021 to the University of Virginia—funding the National Center for Research on Early Childhood Education (NCRECE). The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. Correspondence concerning this manuscript can
References (43)
- et al.
Program integrity in primary and early secondary prevention: are implementation effects out of control?
Clinical Psychology Review
(1998) - et al.
Building support for language and early literacy in preschool classrooms through in-service professional development: effects of the Literacy Environment Enrichment Program (LEEP)
Early Childhood Research Quarterly
(2007) - et al.
Ready to learn? Children's pre-academic achievement in pre-kindergarten programs
Early Childhood Research Quarterly
(2008) - et al.
Using coaching to increase preschool teachers' use of emergent literacy teaching strategies
Early Childhood Research Quarterly
(2009) - et al.
Implementation of a relationship-based school readiness intervention: a multidimensional approach to fidelity measurement for early childhood
Early Childhood Research Quarterly
(2010) - et al.
Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms
Early Childhood Research Quarterly
(2008) - et al.
The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations
Journal of Personality and Social Psychology
(1986) - et al.
The role of mental health factors and program engagement in the effectiveness of preventive parenting program for head start mothers
Child Development
(2003) - et al.
Executive functions and school readiness intervention: impact, moderation, and mediation in the Head Start REDI program
Development and Psychopathology
(2008) - et al.
Putting the pieces together: an integrated model of program implementation
Prevention Science
(2011)
Effects of a parent-child communications intervention on young adolescents' risk for early onset of sexual intercourse
Family Planning Perspectives
Predicting child outcomes at the end of kindergarten from the quality of pre-kindergarten teacher–child interactions and instruction
Applied Developmental Science
Trainers' behavior and participants' persistence in a longitudinal preventive intervention for disruptive boys
journal of Primary Prevention
Effects of a preschool mathematics curriculum: summative research on the Building Blocks project
Journal for Research in Mathematics Education
Individual and organizational trust in a reciprocal peer coaching context
Mentoring and Tutoring: Partnership in Learning
The study of implementation: current findings from effective programs that prevent mental disorders in school-aged children
Journal of Educational and Psychological Consultation
Fostering high-quality teaching with an enriched curriculum and professional development support: the Head Start REDI Program
American Education Research Journal
Teacher characteristics associated with responsiveness and exposure to consultation and online professional development resources
Early Education & Development
Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation
American Journal of Community Psychology
The analysis of intervention change in children and families: methodological and conceptual issues embedded in intervention studies
Journal of Abnormal Child Psychology
Cited by (20)
Patterns of preschool educators' beliefs and practices over the course of a coaching intervention
2022, Journal of School PsychologyCitation Excerpt :It motivates offering professional supports not just reactively to struggling teachers, but also proactively to strong teachers. Importantly, educational leaders making decisions about who to offer professional supports should consider initial educator beliefs and practices in conjunction with the range of other educator, school, and intervention characteristics also shown relevant for these supports' efficacy (e.g., Hanno, McCoy, et al., 2021; LoCasale-Crouch et al., 2016; Sabol et al., 2021; Sebastian et al., 2019; Tolan et al., 2020). Finally, there was little evidence that educator profiles at follow-up were relevant for children's development, suggesting that professional development-induced improvements in educator outcomes may not necessarily translate into immediate improvements in child outcomes.
Personal and contextual factors associated with growth in preschool teachers’ self-efficacy beliefs during a longitudinal professional development study
2018, Teaching and Teacher EducationCitation Excerpt :In addition, we explored whether the two professional development elements may have moderated the impact of efficacy-building experiences and classroom context factors on teachers' self-efficacy beliefs. Participants included 341 preschool teachers drawn from a larger sample of the NCRECE study (Hamre et al., 2012; LoCasale-Crouch et al., 2016; Pianta et al., 2017). The NCRECE study was approved by the last author's university's institutional review board.
Linguistic environment of preschool classrooms: What dimensions support children's language growth?
2018, Early Childhood Research QuarterlyCitation Excerpt :The present results suggest that systems-level classroom quality measures may not adequately document the critical proximal processes that most contribute to children’s development in preschool settings. In this regard, the present study also raises questions regarding some current directions for how to improve children’s language development in the preschool setting, specifically those which focus on improving the systems-level general environment through intensive professional development of teachers (Early, Maxwell, Ponder, & Pan, 2017; Hamre et al., 2012; LoCasale-Crouch et al., 2016). The outcome of these studies is the CLASS observational tool, presumably because improvements in the systems-level general quality of the classroom environment will have downstream effects on children’s language skills, among other cognitive outcomes (LoCasale-Crouch et al., 2016).
Effects of an early childhood educator coaching intervention on preschoolers: The role of classroom age composition
2018, Early Childhood Research QuarterlyCitation Excerpt :In addition, a member of the NCRECE staff completed a 23-item fidelity checklist several times throughout the year for each coach, with total scores representing the percentage of the intervention components completed by a coach. Based on these ratings by the NCRECE staff, coaches averaged a fidelity rate of 94% (range 91–97%; LoCasale-Crouch et al., 2016). Teachers also rated the quality of their experiences with their coaches several times a year with 10 survey items based on a 4-point scale, and on average, rated their coaches quite highly (M = 3.50, SD = 0.41; Pianta et al., 2014).