Elsevier

Early Childhood Research Quarterly

Volume 36, 3rd Quarter 2016, Pages 201-209
Early Childhood Research Quarterly

Unpacking intervention effects: Teacher responsiveness as a mediator of perceived intervention quality and change in teaching practice

https://doi.org/10.1016/j.ecresq.2015.12.022Get rights and content

Highlights

  • Teachers’ perception of intervention quality relates to how responsive they are in both the course and coaching interventions.

  • In the coaching intervention, but not in the course intervention, responsiveness mediates the perception of intervention quality and change in teachers’ instructional interactions.

  • Responsiveness within interventions represents an important and understudied construct in unpacking how and why interventions work.

Abstract

Preschool teachers were randomly assigned to participate in two professional development interventions over two phases, both designed to improve their interactions with children: the NCRECE college course (N = 169) and MyTeachingPartner video-based coaching (N = 202). Using Berkel et al.’s (2011) integrated model of intervention implementation, we examined whether the association between teachers' perception of intervention quality and their improvement in instructional interactions was mediated by their responsiveness to the intervention. We tested the model across both professional development opportunities. Findings indicated that preschool teachers were more responsive to the intervention when they reported higher perceived quality for both the course and coaching. The association between perceived intervention quality and improvement in instructional teaching practice was mediated by teacher responsiveness in the coaching intervention only. Findings indicate that professional development implementers need to focus on how participants perceive interventions and what exactly the key aspects within the interventions are, given the importance of their perspective to what they actually do within interventions that lead to change.

Section snippets

Professional development to improve early childhood teacher–child interactions

Recent work suggests that PD directly targeting improvements in preschool teacher-child emotional, organizational, and instructional interactions can be effective in improving these behaviors (Bierman et al., 2008; Clements & Sarama, 2007; Domitrovich et al., 2009, Hsieh et al., 2009, Pianta La Paro et al., 2008, Pianta Mashburn et al., 2008, Raver et al., 2008). Most of these empirically supported PD approaches provide some combination of curriculum training and classroom-based coaching to

Unpacking what happens within interventions

Recent reviews of intervention evaluation studies note the progress made in conducting field-based randomized trials as a primary mechanism for determining impact, but also call for going beyond understanding if an intervention works to the unpacking of “how” and why” it works (Berkel, Mauricio, Schoenfelder, & Sandler, 2011; Dane & Schneider, 1998; Domitrovich & Greenberg, 2000; Durlack & DuPre, 2008; Irwin & Supplee, 2012). Some of these questions pertain to implementation. Increased

Intervention implementation fidelity and the NCRECE interventions

As outlined by Fixsen, Naoom, Blase, Friedman, and Wallace (2005) in their synthesis of the implementation research literature, intervention implementation should ideally pay attention to multiple facets of fidelity. Most implementation research has focused specifically on fidelity to the model, which typically involves verifying that key intervention components were delivered as intended (Durlak & DuPre, 2008). Fidelity to the model tends to be assessed in a straightforward manner, often with

New directions in intervention implementation research: the role of perceived quality of delivery and participant responsiveness

Although much previous implementation research has focused on fidelity, a growing interest in the quality of delivery is emerging as an understudied aspect of implementation (Berkel et al., 2011; Durlak & DuPre, 2008). Intervention quality of delivery comprises an assessment of whether the core intervention components were delivered in a clear, comprehensible, and enthusiastic manner (Berkel et al., 2011). Evidence suggests that participants' perception of intervention quality may provide a

Current study

Using Berkel et al. (2011) model as the frame, the current study examines the associations among early childhood teachers' perceived quality of delivery, their responsiveness to the intervention across several indicators, and subsequent improved instructional interactions. More specifically, the research questions for this study are: (1) To what extent does teachers' perception of the intervention quality of delivery relate to several aspects of their responsiveness (attendance, active

NCRECE overview and participants

The NCRECE Professional Development Study was a randomized, controlled evaluation of two forms of PD support designed to improve preschool teachers' interactions with children. The study involved a design that included two phases, chronologically ordered. In the first study phase, teachers either were randomly assigned to take the NCRECE course delivered over 14 weeks or were in a control group. Then, teachers were re-randomized and either participated in MyTeachingPartner coaching (Pianta La

Perceived intervention quality of delivery

Teachers were asked to report their perceptions of the course instructor and coach quality of delivery. Items came from previous work that found the scale related to participation in a coaching intervention (Downer, LoCasale-Crouch, Hamre, & Pianta, 2009) and were slightly reworded for this study so they could be used in the course intervention. Scale items were completed outside of the course/coaching on a survey provided by the researchers. Following completion of the course, participants

Engagement

Instructors and coaches completed surveys at the end of the intervention regarding the extent to which teachers were engaged during the intervention. Items came from previous work examining teacher engagement within MTP coaching (Downer et al., 2009) and were slightly reworded for this study so they could be used in the course intervention as well. Instructors responded to 10 items on a 5-point scale (Cronbach's alpha = .96). Items included, “This teacher brought his/her own ideas and questions

Instructional interactions

The effectiveness of teacher-child interactions was rated using the Classroom Assessment Scoring System (CLASS; Pianta La Paro et al., 2008, Pianta Mashburn et al., 2008). The CLASS is an observational assessment of global teacher-child interactions in classroom settings. Each dimension is rated on a 7-point scale with behavioral indicators and anchor point descriptions provided for low, medium, and high levels of that dimension. For the purposes of this study, we examined the CLASS

Results

Our primary goal was to determine whether teachers' perceptions of the intervention quality of delivery were related to their responsiveness to the intervention, and whether responsiveness, in turn, was related to changes in their instructional support (see Fig. 1). Given our interest in testing responsiveness as an intervening variable in the relation between quality and changes in instructional support, we tested our model using mediation analysis (Baron & Kenny, 1986). We examined separate

Course

The bivariate correlations between the variables included in the course phase mediation model are presented in Table 2. These correlations suggest that perception of quality of delivery is significantly related to all three of the responsiveness measures, but neither quality of delivery nor the measures of responsiveness have a significant bivariate relation with posttest instructional support scores. Further, given the low to moderate correlations among the different measures of

Coaching

The bivariate correlations between the variables included in the coaching phase mediation model are presented in Table 4. These correlations suggest that quality is marginally related to coach-rated engagement (p = .06) and significantly related to teacher attendance (i.e., number of cycles and time spent using the web). Rated engagement and the number of cycles (attendance) were significantly related to posttest instructional support scores, but the time spent using web resources (home practice)

Discussion

The present study aimed to use a theoretical framework not previously tested (Berkel et al., 2011) to increase understanding of the ways in which implementation of two interventions (the NCRECE course and MTP coaching) already demonstrated in intent-to-treat analyses to improve the quality of teacher-child interactions (Downer et al., 2012, Hamre et al., 2012) might work. In both interventions explored, responsiveness related to change in teachers' instructional interactions and teachers'

Acknowledgements

The authors wish to thank the generous programs and teachers who participated in this study. The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A060021 to the University of Virginia—funding the National Center for Research on Early Childhood Education (NCRECE). The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. Correspondence concerning this manuscript can

References (43)

  • S.M. Blake et al.

    Effects of a parent-child communications intervention on young adolescents' risk for early onset of sexual intercourse

    Family Planning Perspectives

    (2001)
  • M. Burchinal et al.

    Predicting child outcomes at the end of kindergarten from the quality of pre-kindergarten teacher–child interactions and instruction

    Applied Developmental Science

    (2008)
  • P. Charlebois et al.

    Trainers' behavior and participants' persistence in a longitudinal preventive intervention for disruptive boys

    journal of Primary Prevention

    (2004)
  • D.H. Clements et al.

    Effects of a preschool mathematics curriculum: summative research on the Building Blocks project

    Journal for Research in Mathematics Education

    (2007)
  • E. Cox

    Individual and organizational trust in a reciprocal peer coaching context

    Mentoring and Tutoring: Partnership in Learning

    (2012)
  • C.E. Domitrovich et al.

    The study of implementation: current findings from effective programs that prevent mental disorders in school-aged children

    Journal of Educational and Psychological Consultation

    (2000)
  • C.E. Domitrovich et al.

    Fostering high-quality teaching with an enriched curriculum and professional development support: the Head Start REDI Program

    American Education Research Journal

    (2009)
  • J.T. Downer et al.

    Teacher characteristics associated with responsiveness and exposure to consultation and online professional development resources

    Early Education & Development

    (2009)
  • Downer, J., Pianta, R., Burchinal, M., Field, S., Hamre, B., LoCasale-Crouch, J., … Scott-Little, C. (2012). Coaching...
  • J.A. Durlack et al.

    Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation

    American Journal of Community Psychology

    (2008)
  • J.M. Eddy et al.

    The analysis of intervention change in children and families: methodological and conceptual issues embedded in intervention studies

    Journal of Abnormal Child Psychology

    (1998)
  • Cited by (20)

    • Patterns of preschool educators' beliefs and practices over the course of a coaching intervention

      2022, Journal of School Psychology
      Citation Excerpt :

      It motivates offering professional supports not just reactively to struggling teachers, but also proactively to strong teachers. Importantly, educational leaders making decisions about who to offer professional supports should consider initial educator beliefs and practices in conjunction with the range of other educator, school, and intervention characteristics also shown relevant for these supports' efficacy (e.g., Hanno, McCoy, et al., 2021; LoCasale-Crouch et al., 2016; Sabol et al., 2021; Sebastian et al., 2019; Tolan et al., 2020). Finally, there was little evidence that educator profiles at follow-up were relevant for children's development, suggesting that professional development-induced improvements in educator outcomes may not necessarily translate into immediate improvements in child outcomes.

    • Personal and contextual factors associated with growth in preschool teachers’ self-efficacy beliefs during a longitudinal professional development study

      2018, Teaching and Teacher Education
      Citation Excerpt :

      In addition, we explored whether the two professional development elements may have moderated the impact of efficacy-building experiences and classroom context factors on teachers' self-efficacy beliefs. Participants included 341 preschool teachers drawn from a larger sample of the NCRECE study (Hamre et al., 2012; LoCasale-Crouch et al., 2016; Pianta et al., 2017). The NCRECE study was approved by the last author's university's institutional review board.

    • Linguistic environment of preschool classrooms: What dimensions support children's language growth?

      2018, Early Childhood Research Quarterly
      Citation Excerpt :

      The present results suggest that systems-level classroom quality measures may not adequately document the critical proximal processes that most contribute to children’s development in preschool settings. In this regard, the present study also raises questions regarding some current directions for how to improve children’s language development in the preschool setting, specifically those which focus on improving the systems-level general environment through intensive professional development of teachers (Early, Maxwell, Ponder, & Pan, 2017; Hamre et al., 2012; LoCasale-Crouch et al., 2016). The outcome of these studies is the CLASS observational tool, presumably because improvements in the systems-level general quality of the classroom environment will have downstream effects on children’s language skills, among other cognitive outcomes (LoCasale-Crouch et al., 2016).

    • Effects of an early childhood educator coaching intervention on preschoolers: The role of classroom age composition

      2018, Early Childhood Research Quarterly
      Citation Excerpt :

      In addition, a member of the NCRECE staff completed a 23-item fidelity checklist several times throughout the year for each coach, with total scores representing the percentage of the intervention components completed by a coach. Based on these ratings by the NCRECE staff, coaches averaged a fidelity rate of 94% (range 91–97%; LoCasale-Crouch et al., 2016). Teachers also rated the quality of their experiences with their coaches several times a year with 10 survey items based on a 4-point scale, and on average, rated their coaches quite highly (M = 3.50, SD = 0.41; Pianta et al., 2014).

    View all citing articles on Scopus
    View full text