Does attending a state-funded preschool program improve letter name knowledge?

Highlights

• This study investigated the impact of attending a state-funded pre-K program on letter name knowledge using a regression discontinuity (RD) design.

• Children who attended pre-K (n = 9,689) had higher letter name knowledge (9 letters higher) compared to students who had just begun pre-K (n = 10,897).

• Effect sizes were large (ES = 0.89–1.01) and similar to other statewide pre-K evaluations using an RD design with a similar outcome.

Abstract

This study investigated the causal impact of attending a state-funded pre-K program, the Virginia Preschool Initiative (VPI), on letter name knowledge using a regression discontinuity (RD) design. Children who attended VPI (n = 9,689) had higher letter name knowledge (9 letters higher) compared to students who had just begun VPI (n = 10,897). Findings were robust across various model specifications and imputation methods used. Effect sizes were large (ES = 0.89–1.01) and comparable to other statewide pre-K evaluations using an RD design with a similar outcome.

Introduction

The preschool years are viewed by many as an important time for children to gain prerequisite skills that foster and support future literacy development (Sayeski, Burgess, Pianta, & Lloyd, 2001). One skill in particular, letter name knowledge, has often been used to assess a child for future reading difficulty and to predict future reading achievement (Adams, 1994; Snow, Burns, & Griffin, 1998). Although the benefits of the attendance of preschool programs on cognitive skills seem self-evident given that much of early preschool instruction is focused on building early language and literacy skills (Connor, Morrison, & Slominski, 2006), the causal evidence to support such claims is limited but growing. For example, a rigorously designed study that investigated the effects of preschool attendance in Oklahoma (n = 838) showed a small positive effect size (ES) for receptive vocabulary skills (ES = 0.29) but did not find statistically significant effects for both math and print awareness skills (Wong, Cook, Barnett, & Jung, 2008). In an earlier review of the effectiveness of Head Start, researchers reported that Head Start students knew about 4 letters in the fall and left with knowing about 9 letters in spring though the gain in the Letter-Word Identification task (Woodcock & Mather, 2000) was small (ES = 0.05) and not statistically significant (Zill et al., 2006; p. 2–7). In a review of four studies that estimated the effects of preschool and kindergarten programs on alphabet knowledge (which includes letter name knowledge), the National Early Literacy Panel (NELP; 2008) found only a small effect size (ES = 0.28, 95% CI = −0.18–0.64) which was also not statistically significant (p = 0.27). Despite various early learning initiatives, little is known about the effect of early instruction on alphabet knowledge (Piasta & Wagner, 2010).

The objective of the current study was to evaluate the causal relationship of attending a state-funded preschool program on children’s letter name knowledge scores using a regression discontinuity (RD) design. Much of the recent evidence on the causal, short-term effects of public preschool programs have used an RD design as a result of the design’s strong internal validity when RD assumptions are met (Lipsey, Weiland, Yoshikawa, Wilson, & Hofer, 2014). In the past decade, a growing number of state-funded pre-K programs been evaluated using RD designs (e.g., Gormley & Gayer, 2005; Lipsey, Farran, Bilbrey, Hofer, & Dong, 2011; Wong et al., 2008).

Being able to identify the letters of the alphabet by name is an essential foundational skill in early literacy development (Adams, 1994; Treiman, Kessler, & Pollo, 2006). Understanding that the alphabet is a symbolic system that represents speech sounds is an important stage in a child’s literacy growth (Foulin, 2005, Whitehurst and Lonigan, 1998). The ability to name letters in the alphabet in preschool and kindergarten has historically been identified to be one of the best predictors of future reading ability (Bond & Dykstra, 1967; Catts, Fey, Tomblin, & Zhang, 2002; Foulin, 2005, Hammill, 2004; Schatschneider, Fletcher, Francis, Carlson, & Foorman, 2004; Share, Jorm, Maclean, & Matthews, 1984).

Poorly developed letter name knowledge has consistently shown to be reliable predictor of later reading difficulty (McCardle, Scarborough, & Catts, 2001; Snow et al., 1998). On its own, letter name knowledge can be as effective at predicting later reading skills compared to administering an entire reading readiness test (Scanlon and Vellutino, 1996, Scarborough, 1998, Snow et al., 1998). The predictive ability of alphabet knowledge has been found to be independent of a child’s socioeconomic status, IQ, age, and other early literacy skills such as phonological awareness (NELP; 2008; Snowling, Gallagher, & Frith, 2003; Storch & Whitehurst, 2002).

Furthermore, the relationship between letter name knowledge and later literacy skill is likely causal in nature (Kim, Petscher, Foorman, & Zhou, 2010; Share, 2004, Treiman and Kessler, 2003). Familiarity with letter names provides children a useful mnemonic peg to use while learning about the different visual and auditory characteristics of the letters (Adams, 1994). Learning letter names may help children in learning letter sounds which benefits overall reading (Treiman, Tincoff, & Richmond-Welty, 1997). Studies have shown that children are able to use sound cues embedded at the start (e.g.,/p/in P) or end (e.g.,/es/in S) of the English letter names to learn letter sounds (Evans, Bell, Shaw, Moretti, & Page, 2006; Huang and Invernizzi, 2014, McBride-Chang, 1999). In addition, letter name knowledge is used by young children in connecting both printed and spoken words (Treiman & Rodriguez, 1999). Knowledge of letter names helps facilitate the development of the alphabetic principle or the understanding that patterns of letters represent sounds of spoken language (Huang, Tortorelli, & Invernizzi, 2014). Overall, findings of studies over the decades indicate that children with higher letter name knowledge have a greater likelihood of experiencing success in literacy learning compared to students with lower letter name knowledge who may be at risk of future reading difficulties (Piasta, Petscher, & Justice, 2012).

The importance of early letter name knowledge is further highlighted given that both state and federal preschool-related programs focus on letter name knowledge as a formal target. The federally funded Early Reading First program set performance goals that preschoolers should register growth in letter name knowledge and grantees were required to submit annual reports to document their progress (U.S. Department of Education, 2009). The more recently developed Common Core State Standards also set letter name targets that by the end of kindergarten, children are expected to recognize all the upper and lowercase letters of the alphabet (Council of Chief State School Officers & the National Governors Association, 2010). State preschool programs have varied benchmarks for letter name knowledge ranging from formally knowing 10–20 letters by the end of preschool or more loosely being able to recognize some or several letters of the alphabet (see Piasta et al., 2012 for a detailed account of various benchmarks). A large body of evidence suggests that a child’s early language and literacy experiences in preschool can have beneficial long-term effects and preschool experience can facilitate alphabet knowledge growth (Connor et al., 2006).

Approximately 1.1 million four-year-old children attended one of the 40 state-funded preschool programs in 2014 (Barnett, Carolan, Fitzgerald, & Squires, 2012). However, only 3 out of 10 four-year-olds are enrolled in a high quality preschool program and President Obama has set a goal of enrolling six million children in high quality preschools by the end of the decade (Slack, 2013). Given the recent increases in budgets for preschool programs and renewed attention on preschool initiatives, evidence on the causal effects of preschool are essential in order to justify the continued support for the expansion and improvement of preschool initiatives.

The benefits of preschool attendance have been well documented for decades with experimental studies such as the HighScope Perry Preschool Project (Schweinhart and Weikart, 1981, Schweinhart, 1994) and the Carolina Abecedarian Project (Campbell and Ramey, 1994, Ramey et al., 2000). These well-known early childhood programs provided services targeted towards families from disadvantaged backgrounds and have found that high quality early childhood care can have an impact on cognitive development, academic achievement, health-related outcomes, and behavior (Conti, Heckman, & Pinto, 2015). Although the benefits of preschool attendance have been well studied, early experimental studies had small sample sizes (e.g., n = 123 for the Perry Preschool Program; n = 111 for the Abecedarian Project), limited geographical reach, and were intensive, multi-year programs which may raise questions about their generalizability when implemented on a large scale such as in the case of one-year, state-funded pre-K programs serving children of various backgrounds (Barnett, 2011).

In addition to the early experimental studies, a sizeable body of nonexperimental studies though has demonstrated that preschool programs can benefit a large number of students (Burchinal, Peisner-Feinberg, Bryant, & Clifford, 2000; Camilli, Vargas, Ryan, & Barnett, 2010; Howes et al., 2008; Pianta, Barnett, Burchinal, & Thornburg, 2009). A review of 20 state-funded pre-K programs noted positive impacts across several domains though several methodological weaknesses in the evaluations were noted and limited the ability to draw solid conclusions about pre-K program effects (Gilliam & Zigler, 2004). One major issue in pre-K effectiveness studies is the lack of a comparison group and the issue of selection bias, wherein the participants of pre-K programs systematically differ from those who did not attend pre-K on both observed and unobserved variables which are then possibly related to the outcomes of interest (Gormley, 2007). However, a growing number of studies have evaluated state-funded pre-K programs using an RD design which today constitutes the primary body of research supporting the short term effects of pre-K programs (Lipsey et al., 2014).

Traditional evaluation designs may compare outcomes of participants who received the treatment against the ones who did not and in the preschool context, studies may involve comparing the outcomes of students in the fall of kindergarten who had gone through preschool vs. the students who had not gone through any preschool program. However, a challenge in such a design is that students who had not gone through any preschool program may differ on a variety of characteristics compared to the students who had been through preschool, resulting in a problem of selection bias. Controlling for observed variables is often done with the inclusion of covariates in the statistical models though unobserved variables that have an association with various outcomes may still be present and/or there may be a lack of overlap with regard to the covariates used. In other words, non-experimental studies that only used statistical controls may still have biased results.

As large scale randomized control trials (RCTs) assigning children to attend preschool or not is not practical or always feasible, researchers have turned to regression discontinuity (RD) designs (Trochim, 1984) to assess the causal impacts of preschool attendance. Next to an RCT, studies using RD designs are considered the next best design for obtaining unbiased causal estimates (Institute of Education Sciences, 2014; Lipsey et al., 2014). Though the RD design has been around since the 1960s (Thistlethwaite & Campbell, 1960), it has only been used more frequently in education and psychology in the past decade (Cook, 2008).

The first study to utilize an RD design to evaluate a large scale pre-K program was in Tulsa, Oklahoma (Gormley & Gayer, 2005; Gormley, Gayer, Phillips, & Dawson, 2005). As a result of Tulsa’s strict age cut-off for pre-K eligibility (children must be 4 years of age by September 1 in the year of enrollment), students (n = 2354) were divided into a treatment group, those who were age 4 by the designated cutoff date and were able to enroll in pre-K, and a control group or those who did not make the age cutoff and had to enroll in pre-K the following year. After a year, the students who had gone through pre-K were assessed using various instruments when they were enrolled in kindergarten in the fall and the students who had to wait a year and had just entered pre-K were also assessed using the same measures. As a result, the students in kindergarten, who had received the pre-K treatment, could be compared to the students who were just starting pre-K or those who had not yet received the full year of treatment. The assumption is that participants who selected to attend pre-K were similar on both unobserved and observed characteristics thereby minimizing selection bias. The findings of the seminal study were that pre-K attendance could boost language scores (d = 0.38), cognitive knowledge (d = 0.39), and motor skills (d = 0.24). However, pre-K attendance did not result in a statistically significant increase for socioemotional development (probably as a result of the constrained range of the measure used).

Since the Tulsa study, several other state pre-K evaluations have used a similar birthdate cutoff RD methodology. A five-state pre-K evaluation (ns = 720–2072) used an RD design in New Jersey, Oklahoma, Michigan, South Carolina, and West Virginia and found variability in treatment effects with the largest gains (weighted d = 0.68) coming from the print awareness task which the authors described as a test of alphabet knowledge (Wong et al., 2008). Other states evaluated by the National Institute for Early Education Research (NIEER) using an RD design included Arkansas (Hustedt, Barnett, Jung, & Thomas, 2007) (n = 1408; d = 0.76 for print awareness) and New Mexico (Hustedt, Barnett, Jung, & Friedman, 2010) (n = 1358; d = 1.30 for early literacy).

An RD study in Georgia (Peisner-Feinberg, Schaaf, LaForett, Hildebrandt, & Sideris, 2014) found large effect sizes for letter naming (d = 1.10; n = 1178) and in North Carolina (Peisner-Feinberg & Schaaf, 2011) large effect sizes were found as well for the Letter-Word Identification (LWI) task (n = 1010; d = 1.14). The LWI task is a subscale of the Woodcock-Johnson III Tests of Achievement (Woodcock & Mather, 2000) where respondents are assessed in identifying letters and words. Another study in Tennessee (Lipsey et al., 2011) also found large effect sizes for LWI (d = 0.82, n = 1225). Other pre-K RD studies in large cities such as Boston (Weiland & Yoshikawa, 2013) and San Francisco (Applied Survey Research, 2013) have shown moderate to large LWI gains as well, n = 2018, d = 0.62 and n = 606, d = 0.40, respectively. In summary, RD studies have generally shown moderate to large and practically meaningful causal effects of preschool attendance on print awareness related skills though some other, non RD-based studies have shown smaller effects (NELP, 2008; Zill et al., 2006).

The objective of the current study was to determine the effect of attendance of a state-funded preschool program, the Virginia Preschool Initiative (VPI), on letter name knowledge. Even though an earlier study investigated the benefits of preschool attendance in Virginia, the study acknowledged that findings were correlational in nature and did not specifically investigate letter name knowledge (Huang, Invernizzi, & Drake, 2012). Despite its foundational role in early literacy development, alphabet knowledge is often overlooked as a specific outcome of interest when compared to other aspects of early literacy instruction (Piasta & Wagner, 2010).

Based on the literature review on the effects of pre-K attendance and the importance of letter name knowledge, we hypothesized that preschool attendance would result in an increase in letter name knowledge. Although several recent studies have investigated preschool effectiveness using an RD design, literally only a handful of statewide pre-K evaluation studies using a rigorous quasi-experimental design have appeared in peer-reviewed journals. In addition, our aim was to replicate past RD studies using a large and diverse statewide sample of students. Replication studies reflect an important trend in science considering that a majority of studies fail to replicate (Bohannon, 2014).

The Virginia Preschool Initiative (VPI) was established in 1995 to provide preschool education to at-risk four-year-olds (Virginia Department of Education [VDOE], 2015a). VPI is a targeted program designed to provide preschool education to children not already served by another early childhood program (e.g., Head Start). Funding for VPI is shared between the Commonwealth of Virginia and the locality; the share paid for by locality is dependent on a composite index based on the localities ability to pay. Students are eligible for VPI if they meet one of the following criteria: poverty, homelessness, or parents or guardians are school dropouts (see http://www.doe.virginia.gov/instruction/early_childhood/preschool_initiative/guidelines.pdf for specifics). In addition, certain local criteria may also be used to identify at risk children (e.g., English language learner, parents are incarcerated, foster care). To qualify for state funding, localities must develop a written plan including five services related to: quality preschool education, parental involvement, comprehensive child health services, comprehensive social services, and transportation.

In VPI classrooms, the maximum class size is 18 students and a full time teacher’s aide will be assigned if the class exceeds nine students. Teachers must have a certification specializing in pre-K education. In school year (SY) 2011–12, the per pupil allocation was $6000 (Barnett et al., 2012). Localities have the flexibility to set teacher compensation levels, classroom setting, and daily schedules. In addition, localities may select curriculum content so long as it aligns with the state’s Foundation Blocks for Early Learning (VDOE, 2013b) standards which states that children must be able to demonstrate basic knowledge of the alphabetic principle and that letters represent sounds in spoken words. Although there is no one prescribed curriculum, in recent years, based on analyses of teacher reported data, the most frequently-used curricula in VPI programs included the High Scope Curriculum (used by 24% of VPI students), Creative Curriculum (15%), Core Knowledge (10%), Splash into Pre-K (9%), Big Day for Pre-K (7%), and Tools of the Mind (5%). Five percent of students had no information on the curriculum used, 8% indicated the publisher only (e.g., Houghton Mifflin Harcourt), and the rest used teacher/district developed curricula or some other commercial program (e.g., Opening the World of Learning).

Based on a report by Smart Beginnings (2013) and using data from the VDOE, in 2012, a total of approximately 16,000 children were enrolled in a VPI program, which included around 5000 students enrolled in programs receiving blended funding or funding from other sources in addition to VPI. However, the state had allocated funds for an estimated 23,000 full-day slots, indicating that 30% of the available slots were left unused. Despite the unused slots, there was a wait-list of almost 4000 students. In a report of the Joint Legislative Audit and Review Commission (JLARC; 2007), the top barriers to implementation of VPI by localities included the inability to meet the required local match funding or the insufficient space in school facilities.

Although there is considerable variation in state-funded pre-K programs (see Wong et al., 2008), the National Institute for Early Education Research (NIEER) produces an annual state preschool yearbook that allows for state-by-state comparisons (Barnett et al., 2012). As of 2012, Virginia ranked 25 in terms of access (out of 40) where 16% of four-year-olds were enrolled (e.g., Oklahoma, which has a universal preschool program, had 74% of four-year-olds enrolled and ranked number two). Based on the quality standards checklist, VPI met six out of ten benchmarks established by NIEER (i.e., VPI did not meet minimum standards for teacher degrees since the lead instructors were not required to be present at all times while classes were in session, VPI did not require site visits, nor were meal services required).