Do better schools help to prolong early childhood education effects?

https://doi.org/10.1016/j.appdev.2019.101092Get rights and content

Highlights

  • Children who attend an early childhood education program are more likely to attend a higher quality school

  • Attending a magnet middle school is associated with higher state test scores

  • Attending a magnet middle school is associated with higher rates of attending magnet high school

Abstract

As scholars investigate factors to prolong early childhood education (ECE) effects on student achievement, a neglected hypothesis is that subsequent school quality promotes this goal. We test this, using data from 1844 students who attended kindergarten in the Tulsa Public Schools (TPS) in 2006 and who were identifiable in the school district a decade later. Approximately half of those students attended an ECE program. We establish a close link between school quality and magnet schools which we use as a proxy for higher quality schools. Using propensity score weighting with multiple imputation, we find that ECE alumni are more likely to attend a magnet middle school and a magnet high school. We find that magnet middle school attendance yields higher test scores, after controlling for multiple variables. We conclude that higher quality middle schools could help school districts to sustain short-term gains from ECE for a diverse cross-section of students.

Introduction

Enthusiasm for early childhood education (ECE) programs has grown in recent years, triggering sharp increases in state funding and enrollment rates for four-year-olds (Barnett et al., 2017). Numerous research studies have validated these investments by documenting substantial improvements in school readiness for students attending high-quality ECE programs (Phillips et al., 2017; Yoshikawa et al., 2013). The research evidence is bolstered by developmental theory, which stresses the critical importance of the early childhood years in the development of cognitive and socio-emotional skills, given the rapid growth of neurons during this period (Shonkoff & Phillips, 2000). The short-term efficacy of high-quality preschool programs is now firmly established.

Scholarly consensus on longer-term effects has proven more elusive. Although several studies following ECE students into elementary school or middle school have documented persistent advantages for ECE alumni (Barnett, Jung, Youn, & Frede, 2013; Bassok & Miller, 2014; Dodge, Bai, Ladd, & Mutschkin, 2017; Gormley, Phillips, & Anderson, 2018), two well-crafted studies have not (Lipsey, Farran, & Durkin, 2018; Puma et al., 2012). There seems to be persistence in some situations and fadeout in other situations, but the field lacks a consensus in predicting when fadeout will occur and how extensive it will be.

In this paper, we ask whether enrollment in a higher quality public school, proxied by magnet schools, is more likely if a student participated in an ECE program as a four-year-old. We also ask whether that helps to prolong the initial ECE advantage. Our laboratory is Tulsa, Oklahoma, where magnet schools are marginally better than other public schools. We argue, in effect, that superior magnet schools should attract ECE alumni and challenge and stimulate them enough to mitigate fade out effects that have often plagued ECE programs. Using a rich data base, we examine the middle school enrollment practices of ECE alumni, overall and among subgroups of students. Next, we ask whether enrollment in a magnet middle school encourages later enrollment in a magnet high school. Finally, we investigate whether enrollment in a magnet school boosts student achievement, after controlling for student and family characteristics.

Virtually every scholar who has studied the longer-term effects of ECE programs has found some evidence of fadeout: either partial or total (Duncan & Magnuson, 2013). For example, studies of school-based pre-K in Tulsa have found large short-term improvements in cognitive skills, followed by some fadeout but also a persistent pre-K advantage in math test scores in both third and seventh grade (Gormley et al., 2018; Hill, Gormley, & Adelstein, 2015). Some scholars have also found a more complicated pattern – e.g., cognitive fadeout followed by the sudden appearance of non-cognitive differences in adolescence or early adulthood or non-cognitive differences that grow while cognitive differences diminish (Deming, 2009; Heckman, Pinto, & Savelyev, 2013). On balance, research supports the proposition that ECE programs produce lasting benefits (Phillips et al., 2017; Yoshikawa et al., 2013), with the national Head Start Impact Study and the Tennessee Voluntary Pre-K study as notable exceptions (Lipsey et al., 2018; Puma et al., 2012).

In an effort to integrate these disparate findings, scholars have offered possible explanations for variations in fadeout across research sites. One popular explanation is ECE program quality (Samuels, 2018). For example, critics of the Tennessee pre-K and national Head Start studies argue that these programs were either below average in quality or highly variable in quality (Chaudry, Morrissey, Weiland, & Yoshikawa, 2017; Mead, 2015). According to this perspective, fadeout is likely to be more dramatic when ECE program quality is relatively low. However, Farran and Lipsey (2015) dispute claims that the Tennessee pre-K program was below average in quality. Unfortunately, these debates have often focused on relatively weak measures of program quality, such as educational inputs rather than classroom observations of teacher-child interactions, which makes it difficult to reach firm conclusions.

A second popular explanation is the elementary school curriculum. According to one line of argument, sustained ECE effects require that elementary school teachers ratchet up their pedagogy so that more advanced ECE alumni learn lots of new material and advance to a higher level of knowledge and understanding (Gormley et al., 2018). If kindergarten, 1st grade, and 2nd grade teachers ignore the “pre-K revolution” and continue to do exactly what they have been doing, then ECE alumni may stagnate and the achievement gap between alumni and non-alumni may narrow considerably. Studies show that redundant elementary school instruction in math can be harmful to high-achieving students (Engel, Claessens, & Finch, 2013). On the other hand, it is not clear that this helps to explain fadeout. In fact, one recent study (Jenkins et al., 2018), using data from Head Start and TRIAD (an early mathematics demonstration program) plus kindergarten and 1st grade instructional content data, found no statistically significant interactions between instructional content and preschool enrollment on student achievement. In the authors' words: “Instructional measures did little to explain fadeout” (Jenkins et al., 2018, p. 366).

A third explanation, neglected by most researchers, is magnet schools. In a study of the Chicago Child Parent Centers program, using path analysis, Reynolds and Ou (2011) found that preschool participants were more likely to attend magnet schools between the ages of 10 and 14 and that magnet school enrollment was associated with a higher high school graduation rate and a lower juvenile arrest rate. It is important to note that the magnet schools in question had “selective enrollment policies that require good school performance and high expectations for success” (Reynolds & Ou, 2011, p. 564). This suggests that high-quality magnet schools, combined with a strong preschool program, could provide some students with a double boost: pre-K creates the initial surge, which a magnet school helps to sustain. Magnet schools that consider academic performance in the admissions process could be especially advantageous to pre-K alumni who excel in elementary school because they would be more likely to accept outstanding students.

A fourth explanation, which might be viewed as a blend of the second and the third, posits that school quality matters (Bailey, Duncan, Odgers, & Yu, 2017), though it undoubtedly takes different forms. School quality is a difficult concept to define. It might manifest itself as a more robust, more challenging curriculum; stronger teacher workforce; or more motivated student body. But as with magnet schools, students in a higher quality education setting could sustain the initial boost from ECE. Within the Tulsa School District, magnet schools are considered to be stronger schools across a variety of metrics; for example, test scores and student-teacher ratios are better in magnet schools. Therefore, we use magnet school status as a proxy for school quality because these schools encompass more than just higher test scores. Next, we discuss the relationship between magnet schools and school quality.

Unlike charter schools, which have received considerable attention in the literature, magnet schools, often established by large urban school districts, have not been a magnet to education researchers (Engberg, Epple, Imbrogno, Sieg, & Zimmer, 2014). That is unfortunate because the number of students attending magnet schools is roughly comparable to the number attending charter schools, which have been studied much more thoroughly (Polikoff & Hardaway, 2017). Magnet school enrollments, at 2.6 million students, are similar to charter student enrollments at 2.7 million (National Center for Education Statistics (NCES), 2016) and exceed Catholic school enrollments, at 1.8 million students (National Center for Education Statistics (NCES), 2016).

The original rationale for magnet schools, dating back to the late 1960s and early 1970s, was to encourage white students to attend schools with a substantial black student population (Frankenberg, Siegel-Hawley, & Orfield, 2008; Rossell, 2005). By offering opportunities to specialize in a popular subject, such as a foreign language, the performing arts, or STEM, school districts reconstituted schools in black neighborhoods to make them more attractive to white students. They then established admissions criteria that departed from the traditional neighborhood school presumption, in pursuit of racial integration. As magnet schools evolved, some school districts sought to attract black students to white neighborhoods, and some school districts advertised magnets as being good all-around rather than being good in one specific curricular area. Some school districts also created hybrid schools, with a magnet program embedded in a regular neighborhood school. From the outset, magnet schools were viewed as integral parts of the public school system, unlike charter schools, which have typically been situated outside the public school system.

Over the years, scholars have asked whether magnet schools boost student achievement, school diversity, or both. In one early study using national data, Gamoran (1996) found that students in public magnet high schools performed better on standardized tests in science, reading, and social studies compared to other public high school students (there were no statistically significant differences in math). Using data from Connecticut, Bifulco, Cobb, and Bell (2009) found that interdistrict transfers to magnet middle schools yielded gains in reading, and interdistrict transfers to magnet high schools yielded gains in reading and math. Racial diversity also improved for transferring students. In a study of 21 schools in 11 school districts, Betts, Kitmitto, Levin, Bos, and Eaton (2015) found that conversion to a magnet school improved racial and ethnic diversity for both traditional magnets (disadvantaged schools that seek to attract more advantaged students) and destination magnets (advantaged schools that seek to attract more disadvantaged students). However, they found no evidence of test score improvements attributable to magnet school status. Similarly, Engberg et al. (2014) found no evidence that magnets improve test scores, though they did find that magnets improve student behavior. However, school quality more generally has been shown to boost higher test scores and graduation rates (Hastings & Weinstein, 2008; Jackson, 2010; Pop-Eleches & Urquiola, 2013).

In a synthesis of 18 studies, Wang, Herman, and Dockterman (2018) reached three conclusions about magnet school effects on student achievement: first, magnet school effects tend to be “minimal or modest” but positive; second, magnet schools seem to produce larger impacts in high school than in elementary school, perhaps because students play a bigger role in affirmatively choosing their high school; and third, magnet schools produce some positive effects that go beyond standardized test scores (e.g., advanced course-taking, graduation rates) and generally produce bigger gains for more disadvantaged students. Although more research needs to be done, results thus far have been fairly encouraging.

The rationale for expanding state-funded pre-K programs has often highlighted the critical importance of boosting the educational performance of low-income students, including students of color. For example, Chaudry et al. (2017, p. 72) defend their blueprint for a sharp expansion of pre-K and Head Start by arguing that we need to “ensure that all children have an equal playing field for succeeding in school.” Targeted pre-K programs promote equal educational opportunity explicitly by directing funds to at-risk students, including students of color and students from low-income households. Universal pre-K programs do this indirectly by offering the prospect of better outcomes through peer effects for disadvantaged students who attend the same pre-K program as middle-class students (Gormley, 2017a; Weiland & Yoshikawa, 2013). This same emphasis on the educational outcomes of the most disadvantaged students can be found in the Elementary and Secondary Education Act, in the No Child Left Behind, and its successor act, Every Student Succeeds Act. All of these statutes sought, albeit in different ways, to reduce the achievement gap between black and white students, between Latino and white students, and between low-SES and high-SES students, which continues to be a source of great concern (Manna, 2011; Reardon & Portilla, 2016). When examining the effects of ECE programs on state outcomes, it is important to pay close attention to subgroup differences, including variations by race and household income (or its surrogate, school lunch eligibility).

Across studies, there is limited agreement on differences in ECE's short-term impact by demographic subgroups (Ladd, 2017). For example, studies suggest larger pre-K effects among low-income children (e.g., Gormley, Gayer, Phillips, & Dawson, 2005; Weiland & Yoshikawa, 2013), though results also suggest favorable effects for middle-income youth. Hispanic students tend to have the most consistently favorable effects, compared with children of other backgrounds. For example, in Tulsa, Hispanic students' favorable pre-K effects were large and effect sizes were more favorable than many other sub-groups of children (Gormley, 2008). The story of black student success from pre-K is not as consistent as that of Hispanic students (Ladd, 2017). Finally, there is some indication that subgroup differences by race may instead be driven by income status, given that children from minority backgrounds are overrepresented in lower income groups, at least at the national level (Bassok, 2010).

Whether fadeout of ECE effects varies by subgroup remains unclear, because many studies of longer-term effects do not report subgroup effects thoroughly (Dodge et al., 2017) or at all (Barnett et al., 2013). A national study of Head Start's longer-term effects found more fadeout for blacks and less fadeout for whites and Hispanics (Deming, 2009). That same study found more fadeout for girls than for boys. Our own studies in Tulsa were inconclusive on gender; on race, we generally saw more fadeout for black and less for Hispanic students (Gormley et al., 2018; Phillips, Gormley, & Anderson, 2016).

Tulsa Public Schools (TPS) is a large, urban school district that is ideal for studying the link between ECE and school quality. At the time of our study, TPS enrolled more students, over 40,000, than any other school district in Oklahoma. It also had a very diverse student body with respect to race-ethnicity, family income, and home language. In terms of our study interests, TPS offers both universal pre-K and Head Start (under a state-sanctioned partnership with the Community Action Project of Tulsa County) to four-year-old children and has magnet middle and high schools. These ECE programs maintain high quality standards, as required by Oklahoma state law. For example, all teachers must have a bachelor's degree and must be early childhood certified; and child/staff ratios of ten-to-one must be maintained.

As noted previously, magnet schools emerged in the late 1960s and the early 1970s as a plausible strategy for promoting racial integration in U.S. schools. In Tulsa, school administrators became interested in the magnet school idea in the early 1970s, after a federal district court judge ruled that TPS was not complying with Brown v Board of Education (1954) or the Civil Rights Act of 1964 and ordered that segregated schools be shut down. In the early 1970s the George Washington Carver Middle School and the Booker T. Washington High School, both located in historic black neighborhoods, reopened as magnets with a mandate to achieve a more favorable racial balance (McDonald, 2015). Over time, additional magnet schools were established, including schools in historic white neighborhoods that were expected to attract black and Latino students.

Unlike some school districts, TPS has both magnet middle schools and magnet high schools. Five of the fifteen middle schools are magnets and three of the ten high schools are magnets. Two of the magnet schools are housed within a standard high school. In addition, TPS has two types of magnet schools: admission and lottery. The admission magnet, which is also referred to as an academic magnet, requires that students apply and are selected on the basis of grades, test scores, attendance, and teacher recommendations. For lottery magnets, students apply and are selected based on a random drawing. For standard middle and high schools, students attend based on their geographic proximity to the school.

As Table A.1 in the appendix suggests, there is a striking positive relationship between magnet school status and school quality in Tulsa, for both middle schools and high schools. The evidence comes from school report cards, issued by the Oklahoma Department of Education for the 2014–15 school year, based on standardized test scores and attendance, not adjusted for student characteristics. Among middle schools, 3/5 magnet schools but 0/8 non-magnet schools received an A, B, or C letter grade. Among high schools, 4/4 magnet schools but 0/7 non-magnet schools received an A, B, or C letter grade. Although some magnets are better than others and some non-magnets are better than others, magnets as a whole are clearly superior to non-magnets in Tulsa, based on objective evidence from the state of Oklahoma (though value-added models might yield different results).

Based on our review of the literature, we had several expectations about the relationship between ECE and school quality, which is operationalized by magnet school status. We break down our expectations based on type of magnet school and process of admission (high school only). Attending either an academic magnet school or a lottery magnet school first requires students to apply to the school. We believed that ECE could be a factor in first applying to a school because of the widespread assumption that magnet schools are more attractive and more competitive academically. We describe the types of magnet schools in turn and use the term magnet school in the manuscript to denote the type of school and also as a designator of higher school quality.

Acceptance into an academic magnet school at both the middle school and high school levels is based in part on student grades and teacher recommendations. Because ECE increases student test scores (Gormley et al., 2018), we expected students enrolled in ECE to be accepted into academic magnets at higher rates.

Because acceptance into a lottery magnet school is based on chance and not on academic merit, we did not expect a direct relationship between ECE and lottery magnet attendance at the middle school or high school levels. However, it was expected that there might be an indirect relationship, with ECE encouraging more applications. Therefore, they might be more likely to have their child enroll in a magnet school.

Magnet schools in Tulsa are generally regarded as among the best of the city's public schools, therefore, we expected students enrolled in these schools to perform better on standardized tests compared to students who did not attend magnet schools.

Section snippets

Study sample

Our sample is from Tulsa, Oklahoma, where we have continued to track approximately 1850 of 4033 students who enrolled in the Tulsa Public Schools kindergarten program in the fall of 2006 and who were still students in the Tulsa Public School System as of 10th grade. Approximately half of our students were enrolled in an ECE program as four-year-olds: most of these students enrolled in the school-based pre-K program, while approximately 200 of these students were enrolled in Head Start just

Measures

We used data collected from three sources: (1) state/district administrative data from 2006/2007, 2013/2014, and 2015/2016 academic years for children enrolled in TPS; (2) parent survey data from children enrolled in TPS as collected in August 2006; and (3) U.S. Census data.

Analytic strategy

In the absence of random assignment to ECE, we selected a propensity score weighting approach to estimate the impact of ECE on magnet school attendance and application. In three recent papers (Gormley et al., 2018; Hill et al., 2015; Phillips et al., 2016), researchers estimated the effects of the Tulsa Public School's (TPS) ECE programs on standardized test scores as of third grade and seventh grade, using propensity score weighting. In this paper, we use a similar analytic strategy given

Results

We report findings below for students overall and for subgroups based on race/ethnicity and school lunch eligibility. Fig. 2 shows the simple breakdown of attendance/application/acceptance to magnet schools by ECE attendance. Without controlling for any covariates, students who attended ECE had higher rates of attending magnet schools than those who did not attend ECE, consistent with our expectation. We were also interested in attendance/application/acceptance for subgroups to determine

Discussion

From an outsider's perspective, what is really needed to sustain the positive effects of ECE over time is high-quality K-12 schools available to students from diverse backgrounds. Whether those schools are traditional schools, magnet schools, or charter schools is a secondary concern. From a student's perspective or a parent's perspective, however, the choice to be made is not whether to enroll in a high-quality school, assuming that can be defined, but whether to enroll in a magnet school,

Limitations

We close with a few words of caution. First, ECE participation was not randomly assigned; thus we employed an alternative strategy, propensity score weighting, for mitigating selection effects. This method helps alleviate bias due to systematic differences in nonexperimental settings by creating comparable groups to improve causal inference (Dehejia & Wahba, 2006; Stuart, 2010). Nonetheless, there may have been unobserved child or family characteristics that contributed to ECE enrollment that

Conclusion

In this paper, we examined whether enrollment in a high quality school can help a student who attended an ECE program at age 4 to sustain the educational benefits of that program. In focusing on school quality, we have implicitly embraced what Bailey et al. (2017) have called the “sustaining environments” perspective on how to make early childhood interventions last over time. Efforts to enhance a student's K-12 experience (inside or outside the classroom) would fit that perspective. For

Funding

This work was supported by the Heising-Simons Foundation [2015-033].

References (61)

  • C. Hill et al.

    Do the short-term effects of a high-quality preschool program persist?

    Early Childhood Research Quarterly

    (2015)
  • D. Phillips et al.

    Inside the pre-kindergarten door: classroom climate and instructional time allocation in Tulsa’s pre-K program

    Early Childhood Research Quarterly

    (2009)
  • D. Bailey et al.

    Persistence and fadeout in the impacts of child and adolescent interventions

    Journal of Research on Educational Effectiveness

    (2017)
  • W.S. Barnett et al.

    The state of preschool 2016: State preschool yearbook

    (2017)
  • W.S. Barnett et al.

    Abbott preschool program longitudinal term effects study: Fifth grade follow-up

    (2013)
  • D. Bassok

    Do black and Hispanic children benefit more from preschool? Understanding differences in preschool effects across racial groups

    Child Development

    (2010)
  • D. Bassok et al.

    Do children benefit from widely-available public preschool? Evidence from florida’s voluntary prekindergarten program

    (2014, November)
  • J. Betts et al.

    What happens when schools become magnet schools?

  • R. Bifulco et al.

    Can interdistrict choice boost student achievement? The case of Connecticut’s Interdistrict magnet school program

    Educational Evaluation and Policy Analysis

    (2009)
  • A. Chaudry et al.

    Cradle to kindergarten: A new plan to combat inequality

    (2017)
  • R.H. Dehejia et al.

    Propensity score-matching methods for nonexperimental causal studies

    The Review of Economics and Statistics

    (2006)
  • D. Deming

    Early childhood intervention and life-cycle skill development: Evidence from head start

    American Economic Journal: Applied Economics

    (2009)
  • K.A. Dodge et al.

    Impact of statewide early childhood programs and policies on educational outcomes in elementary school

    Child Development

    (2017)
  • G. Duncan et al.

    Investing in preschool programs

    Journal of Economic Perspectives

    (2013)
  • J. Engberg et al.

    Evaluating educational programs that have lotteried admission and selective attrition

    Journal of Labor Economics

    (2014)
  • M. Engel et al.

    Teaching students what they already know? The (mis)alignment between mathematics instructional content and student knowledge in kindergarten

    Educational Evaluation and Policy Analysis

    (2013)
  • D. Farran et al.

    The tennessee pre-k study

  • D.C. Farran et al.

    Data-driven improvement in prekindergarten classrooms: Report from a partnership in an urban district

    Child Development

    (2017)
  • E. Frankenberg et al.

    The forgotten choice? Rethinking magnet schools in a changing landscape

    The Civil Rights Project

    (2008)
  • A. Gamoran

    Student achievement in public magnet, public comprehensive, and private city high schools

    Educational Evaluation and Policy Analysis

    (1996)
  • C. Gewertz

    The battle over who gets into elite public high schools

  • W. Gormley

    The effects of Oklahoma’s pre-K program on Hispanic students

    Social Science Quarterly

    (2008)
  • W. Gormley

    Universal vs. targeted pre-kindergarten: reflections for policymakers

  • W. Gormley

    The critical advantage: developing critical thinking skills in school

    (2017)
  • W. Gormley et al.

    The effects of Universal Pre-K on cognitive development

    Developmental Psychology

    (2005)
  • W. Gormley et al.

    The effects of Tulsa’s preschool program on middle school student performance

    Journal of Policy Analysis and Management

    (2018)
  • E.A. Hanushek et al.

    Does peer ability affect student achievement?

    Journal of Applied Econometrics

    (2002)
  • V.S. Harder et al.

    Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research

    Psychological Methods

    (2010)
  • J.S. Hastings et al.

    Information, school choice, and academic achievement: Evidence from two experiments

    The Quarterly Journal of Economics

    (2008)
  • J.J. Heckman et al.

    Understanding the mechanisms through which an influential early childhood program boosted adult outcomes

    American Economic Review

    (2013)
  • View full text