Evaluating academic outcomes of Head Start: an application of general growth mixture modeling

https://doi.org/10.1016/S0885-2006(03)00027-9Get rights and content

Abstract

This study intends to illustrate the utility of general growth mixture modeling (GGMM) for evaluation of early childhood education programs, using a sample of children with Head Start experience. In the first analysis of this study growth mixture modeling (GMM) found that children with Head Start experience had two distinct growth patterns. In the second analysis of this study general growth mixture modeling found that children with two or more years of program participation did not have faster achievement growth, on average, than children with only one year of program participation. This study also found that a gender gap in mathematics and an income gap in reading and mathematics were exclusively exhibited by the children with no preschool experience. Therefore, it was concluded that the Head Start program may be reducing both a gender gap in mathematics and an income gap in reading and mathematics.

Introduction

This study intends to illustrate the utility of general growth mixture modeling (GGMM) for evaluation of early childhood education programs, using a sample of children with Head Start experience. The sampling choice stems from the fact that in March of 1998, the U.S. Congress articulated the need for a methodologically sound national evaluation of the Head Start program. This call was motivated by the realization that national evaluations of the program conducted during the last 30+ years were methodologically flawed and inadequate (United States Congress, Senate, Committee on Labor and Human Resources, & Subcommittee on Children and Families, 1998).

One of the most serious methodological problems was comparing Head Start participants to groups who were not really comparable (especially regarding SES). Inadequate sample sizes across many separate local program evaluations represented another problem. In addition to small sample sizes, these studies could only be generalized to very specific areas of the country. This generalizability problem stems from the fact that the program curriculum varies greatly from one local program to the next due to the flexible nature of the program (i.e., sites are given local control) (Administration for Children and Families, 2001).

It is argued that another limitation of past evaluations of the program involves a statistical assumption. Many methodologies used to evaluate Head Start academic outcomes over time, including traditional growth modeling, have assumed that all growth patterns come from one population and that any predictors in the model have the same influence on the growth factors for all of these patterns. But what if there are a mixture of population growth patterns and subsequent predictors in the model influence them differently? Clearly, traditional models are confining in that they do not recognize that different subgroups within one sample may have different patterns of growth (Muthén, Khoo, Francis, & Boscardin, 2002). For example, if the slopes for two subgroups have opposite signs, traditional methods would demonstrate zero slope for the overall population line so that predictors of potential growth patterns are overlooked (Muthén, Brown, Khoo, Yang, & Jo, 1997).

In response to this limitation of previous methodologies, including traditional growth modeling, Muthén (2001) provided the research community with a new technique which captures the heterogeneity of development through categorical latent variable modeling (it uses both continuous and categorical latent variables). The latent categorical variables in this instance are unobserved class membership (i.e., respondents can be classified into the class that has the highest probability of their membership). In other words, categorical latent variables represent groups of individuals who are comparable within a given category, but who are different across categories.

This new procedure, known as growth mixture modeling (GMM), combines conventional growth modeling with a latent class growth analysis (LCGA). Latent class analyses assume that the population of interest is made up of unobserved (i.e., latent) classes of individuals. LCGA is a traditional procedure used to compare growth patterns between groups by using one outcome variable (measured over many time points) to determine a latent class model, where the latent classes create different growth trajectories (Muthén & Muthén, 1999). LCGA estimates a mean growth curve for each class. For example, one class may start out with high achievement and level off slowly, while another group may begin with low achievement but grow quickly. According to Muthén and Muthén (1999), “the object of the analysis is to estimate the different growth curve shapes and the class probabilities” (p. 8). More generally, each latent class corresponds to a sub-population that has its own set of parameter values (Muthén & Muthén, 1999). The reason LCGA cannot be used in lieu of conventional growth modeling is that, although it estimates a mean growth curve for each class, the variation in the growth factors within each class is assumed to be zero. By combining this procedure with traditional growth modeling, it is possible to estimate mean growth curves for each class of trajectories while also capturing individual variations around these growth curves (Muthén, 2001).

Growth mixture modeling is sometimes referred to as the group-based modeling approach, or more often as simply mixture modeling, and is designed to “identify relatively homogeneous clusters of developmental trajectories by assuming that the population is composed of distinct groups defined by their developmental trajectories” (Nagin, 1999, p. 140). Hence, growth mixture modeling can be viewed as a form of cluster analysis where clusters are obtained as a by-product of the growth analysis, by placing an individual in the cluster for which the individual has the highest posterior probability (Muthén et al., 1997; Muthén, Khoo, & Francis, 1998). With this procedure, both the number of best fitting groups and the posterior probability of group membership for evaluating the precision of group assignments are provided (Nagin, 1999).

In addition, this class membership component specified in the GMM can be linked to structural equation models in a procedure defined by Muthén (2001) as general growth mixture modeling. Using this procedure Muthén et al. (1997) explained that it is possible to test whether different trajectory classes have different relationships to previous circumstances (time-invariant and time-varying covariates), mediators and interventions for the growth process. It is also possible to test whether different trajectory classes have different relationships to consequences (distal outcomes) of the growth process (Muthén et al., 1997).

Utilizing this procedure, which encompasses traditional growth modeling and growth mixture modeling, confirmatory analyses may be conducted, allowing one to test the hypothesis that certain types of people have typical growth trajectories (i.e., a common class membership). Again, traditional growth modeling cannot make such hypotheses because the procedure assumes that any predictors in the model have the same influence on the growth factors for all trajectories. For example, GGMM might find that variation in academic development among low achieving students was influenced by the school environment, while variation in academic development for high achieving students was influenced by home environment (Muthén, 2001). This type of test is not possible with traditional growth modeling. In addition, these two types of development could also have different growth shapes, different concurrent processes and different consequences (Muthén, 2001). For example, unlike traditional growth modeling, GGMM could find that “problematic first-grade academic development had a non-linear shape, co-occurred with the development of aggressive behavior and increased the probability of school dropout” (p. 9).

Muthén et al. (1997) discussed GGMM in relation to a study by Schulenberg, O’Malley, Bachman, Wadsworth, and Johnston (1996). Using the outcome variable of frequency of heavy drinking, two latent trajectory classes were estimated, alcohol dependence and no alcohol dependence (Muthén & Muthén, 2000). They explained that GGMM was able to identify the two-class growth model without using any alcohol dependence information. In other words, GGMM identified the group of alcohol dependents based on an early growth trajectory of drinking patterns.

GGMM also allows the effect of treatment to vary across trajectory class and therefore, gives a very explicit appraisal of treatment effects. It is particularly useful with longitudinal studies of growth over time and with intervention studies where a certain treatment could take effect after different amounts of time for different classes of people, depending on their growth pattern (Muthén et al., 1997). Muthén (2001) provided an example of a GGMM analysis of reading skills through first and second-grade “The class membership probabilities can then be estimated for a new student before the student reaches the end of second-grade using only the subset of the repeated measures available at that point in time” (p. 22). Therefore, the growth mixture analysis has great potential for developmental studies, particularly for intervention studies and in the early detection of problematic development (Muthén et al., 1997).

Using growth mixture modeling the first analysis of this study explores the following research question: Do children with Head Start experience have a mixture of population growth patterns?

If more than one growth trajectory is found for children with Head Start experience, the second analysis will explore whether length of time in the program serves as an adequate criterion in predicting growth trajectory class membership. That is, general growth mixture modeling will be used to test the second research question: Do children with two or more years of program participation have faster achievement growth in reading and mathematics, on average, than children with only one year of program participation?

Section snippets

Data source

The database utilized in this study, called Prospects: The Congressionally Mandated Study of Educational Growth and Opportunity1 and was produced by the U.S. Department of Education for evaluation of the federal Chapter One program. It contains data collected during school years 1991–1994 from nationally representative samples selected from three cohorts of

Analysis 1: GMM of reading and mathematics achievement for the Head Start group

Table 2 contains the GMM achievement BIC values for the Head Start group for each of three analyses, which estimated one-class, two-class and three-class models. The first analysis (estimating a one-class model) simply represented a conventional growth model. It was clear from the BIC values that the two-class model was the best fitting of the three for children with Head Start experience. Furthermore, when the three-class model was estimated, the additional class accounted for only a fraction

Discussion

Children with Head Start experience did have a mixture of population growth patterns in reading and mathematics. The no preschool group also exhibited the same growth patterns, which was surprising considering their higher SES overall. This study provided evidence for the first time, that children with Head Start experience (as well as children without this experience) have heterogeneous academic growth patterns. Past evaluations of Head Start’s academic outcomes have implied that children with

Acknowledgements

I thank Dr. David Kaplan for his continued assistance with the methodology and data analysis for this study. I also thank Drs. Linda and Bengt Muthén who not only pioneered the statistics utilized, but also created the statistical program for its application. In addition, they provided me with much information about both the statistics and the Mplus (Muthén & Muthén, 1998) computer program. I thank Dr. Michael Gamel-McCormick for his extensive feedback on this study in light of his experience

References (16)

  • Abt Associates Inc. (1997, April). Prospects: Student outcomes (final report). Washington, DC: U.S. Department of...
  • Administration for Children and Families, U.S. Department of Health and Human Services. (2001, February 20). Head Start...
  • Guralnick, M. J. (1997). The effectiveness of early intervention. Baltimore, MD:...
  • Muthén, B. (2001). Second-generation structural equation modeling with a combination of categorical and continuous...
  • Muthén, B., Brown, C. H., Khoo, S., Yang, C., & Jo, B. (1997). General growth mixture modeling of latent trajectory...
  • Muthén, B., Khoo, S., & Francis, D. J. (1998). Multi-stage analysis of sequential developmental processes to study...
  • Muthén, B., Khoo, S., Francis, D. J., & Boscardin, C. K. (2002). Analysis of reading skills development from...
  • Muthén, B., & Muthén, L. (1998). Mplus: The comprehensive modeling program for applied researchers: User’s guide. Los...
There are more references available in the full text version of this article.

Cited by (24)

  • Linear-linear piecewise growth mixture models with unknown random knots: A primer for school psychology

    2019, Journal of School Psychology
    Citation Excerpt :

    Piecewise functions, in general, are useful in many educational and psychological research contexts in which development may be best characterized by distinct phases. For instance, in studies of early childhood intervention, children may show relatively faster improvement in the post-intervention follow-up period relative to the pre-intervention period (Kreisman, 2003). In this research scenario, the functional form characterizing the rate of change includes a slower acquisition phase that gives way to a later rapid acquisition phase.

  • A two-step sampling weight approach to growth mixture modeling for emergent and developing skills with distributional changes over time

    2017, Journal of School Psychology
    Citation Excerpt :

    Alternatively, growth mixture modeling (GMM) relaxes the single-population assumption to allow for heterogeneous, unobserved classes of students with variant fixed and random trajectory parameters (intercept and slope; e.g., Ram & Grimm, 2009). For example, a Head Start program study found two distinct patterns of reading and math growth: (a) a majority class that displayed below-average initial reading and math scores and exhibited negative growth over time in both subjects, and (b) a minority class that displayed below-average initial reading and math scores, but exhibited steady positive growth (Kreisman, 2003). These groups were not known to the researcher before the study, rather the two heterogeneous, unobserved classes were identified post-hoc through GMM.

View all citing articles on Scopus
View full text