Validating Virginia's quality rating and improvement system among state-funded pre-kindergarten programs☆
Section snippets
Background on Virginia's QRIS: The Virginia Star Quality Initiative (VSQI)
Virginia's QRIS, the Virginia Star Quality Initiative, offers a useful setting for QRIS validation work as it defines ratings using quality standards with strong empirical support, uses a unique rating structure to create star ratings, and is a relative newcomer to the QRIS landscape. Piloted in 2007 and officially implemented in 2008, the main goal of the VSQI is to provide a consistent way to distinguish the level of quality in early education programs within the Commonwealth of Virginia.
Validation of QRIS rating system and quality standards
There are many potential outcomes of interest of QRIS, such as providing a quality framework, unifying early childhood education programs under one system, or promoting professionalization of the early educator workforce. This paper focuses on the ways in which QRIS ratings relate to children's development, building off the assumption that child outcomes are a key outcome of QRIS.
Past research provides little evidence on whether we may expect that the aggregate star ratings relate to child
Using quality standards in a QRIS context
Although there is a general consensus that certain quality standards relate to child outcomes, albeit often modestly, there is little understanding of whether there are certain points in which the magnitude of associations are greater, or points in which associations begin to level off either at the low or high end of the quality distribution (Burchinal, Vandergrift, Pianta, & Mashburn, 2010). This lack of evidence is concerning given that QRIS apply cut-points to individual quality standards
A focus on pre-kindergarten programs
In the current study, we only focus on children who attended Virginia's state-funded targeted pre-kindergarten program, the Virginia Preschool Initiative. Virginia's targeted pre-kindergarten program provides education to at-risk children who are not served by another program (e.g., Head Start). A child is eligible to participate in the pre-kindergarten if they have at least one demographic risk, including if the child lives in poverty, is homeless, comes from a family with low levels of
The current study
The rapid proliferation of QRIS has led to a need for empirical evaluation of how QRIS operate within a potentially high stakes policy context. The QRIS ratings are the key output of QRIS, yet there is very little empirical evidence on whether states’ rating structures capture unique aspects of quality that relate to children's outcomes. The present study validates Virginia's pilot QRIS among children who attended state-funded, targeted pre-kindergarten programs. We capitalize on data collected
Sample description
Analyses draw upon a database constructed from the following datasets: (1) child care facility information from the Virginia Department of Social Services; (2) block group census data from the 2000 U.S. Census; (3) quality standard scores and star ratings on the VSQI from 2007 to 2009; and (4) pre-kindergarten and kindergarten performance on the Phonological Awareness Literacy Screening (PALS), and center and child characteristics from the Virginia Department of Education and the University of
star quality ratings
The VSQI assessed center quality on four performance standards: (1) staff education and qualifications; (2) teacher-child interactions; (3) structure (i.e., staff-to-child ratio); and (4) environment and instruction. Star Quality Raters assessed classrooms through documentation and observations. Star Quality Raters were extensively trained and were tested for inter-rater reliability once every seven visits. In 2007–08 (VSQI Year 1), Star Raters observed one out of every three classrooms. In
Analytic plan
A central tenet to developmental theory is that individual children's development is strikingly heterogeneous across time. Examining the association between child care quality and children's performance using two time points (e.g., predicting fall kindergarten performance while controlling for fall pre-kindergarten performance) does not account for the variability in growth within individual children. Indeed, there is substantial evidence that accounting for within-child variability yields a
Differences between center characteristics
Table 1 presents comparisons of center characteristics among 2-star, 3-star, and 4-star pre-k programs. One-way between-subjects ANOVAs indicated a number of differences between 2-star, 3-star, and 4-star pre-k programs with regard to pre-k programs’ neighborhood characteristics. Two-star programs tended to be located in areas with more African American residents and more single-mother households compared to 3-star and 4-star programs. Four-star programs, and to a lesser extent 3-star programs,
Discussion
Quality Rating and Improvement Systems are an increasingly popular policy approach to assess and improve the quality of early childhood education within a state context. Quality ratings are the central tenant of the QRIS; parental selection, improvement supports and financial incentives are posited to have strong ties to programs’ performance in the rating system. Yet, there is little empirical evidence on whether the complex weighting and aggregation structure used to derive programs’ ratings
Future directions
Although we find some traction that pre-k star ratings relate to growth in the pre-kindergarten year, these relations fade out relatively quickly. The central question remains whether there is a better way to combine and weight the quality standards that predict development across early and later schooling. The aggregation technique in the VSQI, where multiple indicators of quality are summed together, may explain part of the reason that relation among VSQI ratings and outcomes fades over time.
Conclusion
Quality Rating and Improvement Systems have dramatically expanded over the last decade in response to the growing need to improve the quality of early childhood education programs and outcomes for children. Despite the strong theoretical model for QRIS, there is little descriptive understanding of the issues related to QRIS or the alignment of QRIS ratings to empirical evidence of their effects. The evidence that higher star ratings are related to stronger growth in children's pre-literacy
Acknowledgments
The authors are grateful for the guidance and support of this study provided by the Virginia Early Childhood Foundation, the Virginia Office of Early Childhood Development, and the Virginia Department of Education. We would like to acknowledge Deborah Jonas, Zelda Boyd, Maria Brown, Kathy Glazer, Jason Downer, Andrew Mashburn, Amanda Williford, and Yu Cao for their contributions to our work. The research reported here is supported by the Child Care Research Scholars Grant, Administration for
References (59)
- et al.
Diversity, child care quality, and developmental outcomes
Early Childhood Research Quarterly
(2003) - et al.
Piecewise growth curve modeling approach for longitudinal prevention study
Computational Statistics & Data Analysis
(2004) - et al.
Teacher credentials and student achievement: Longitudinal analysis with student fixed effects
Economics of Education Review
(2007) - et al.
Are teachers’ education, major, and credentials related to classroom quality and children's academic gains in pre-kindergarten?
Early Childhood Research Quarterly
(2006) - et al.
Ready to learn? Children's pre-academic achievement in pre-kindergarten programs
Early Childhood Research Quarterly
(2008) - et al.
The differential effects of preschool: Evidence from Virginia
Early Childhood Research Quarterly
(2012) - et al.
How much is too much? The influence of preschool centers on children's social and cognitive development
Economics of Education Review
(2007) - et al.
A multilevel model of child- and classroom-level psychosocial factors that support language and literacy resilience of children in Head Start
Early Childhood Research Quarterly
(2012) - et al.
Teacher qualifications, classroom practices, family characteristics and preschool experience: Complex effects on first graders’ vocabulary and early reading outcomes
Journal of School Psychology
(2005) - et al.
Effects of web-mediated professional development resources on teacher–child interactions in pre-kindergarten classrooms
Early Childhood Research Quarterly
(2008)
The effects of state pre-kindergarten programs on young children's school readiness in five states
The state of preschool 2012: State preschool yearbook
The state of preschool 2007: State preschool yearbook
Do Black and Hispanic children benefit more from preschool? Understanding differences in preschool effects across racial groups
Child Development
How well do our measures of quality predict child outcomes? A meta-analysis and coordinated analysis of data from large-scale studies of early childhood settings
Threshold analysis of association between child care quality and child outcomes for low-income children in pre-kindergarten programs
Early Childhood Research Quarterly
Regression models: Censored, sample selected or truncated data
Validating North Carolina's 5-Star child care licensing system
Meta-analysis of the effects of early education interventions on cognitive and social development
Teachers College Record
Tools for analyzing multiple imputed datasets
Stata Journal
Promoting language and literacy in young dual language learners: Research, practice and policy
Child Development Perspectives
How does your kindergarten classroom affect your earnings? Evidence from Project STAR
Quarterly Journal of Economics
Class size and student outcomes: Research and policy implications
Journal of Policy Analysis and Management
The effects of summer vacation on achievement test scores: A narrative and meta-analytic review
Review of Educational Research
Experimental evidence on the effect of childhood investments on postsecondary attainment and degree completion (No. w17533)
Analyzing longitudinal data with missing values
Rehabilitation Psychology
Children, schools, and inequality
Selection in a field experiment with voluntary participation
Journal of Applied Econometrics
The effects of universal pre-k on cognitive development
Developmental Psychology
Cited by (0)
- ☆
Research contained herein was conducted using data provided under a contract with the Virginia Department of Education. The content does not necessarily reflect the views or polices of the Virginia Department of Education, the Virginia Board of Education, or the Commonwealth of Virginia. Consequently, the Virginia Department of Education, the Virginia Board of Education, and the Commonwealth of Virginia are not responsible for the research brief's content or any loss suffered due to the use of such content. Moreover, the mention of any trade names, commercial products or organizations in this research brief is not an endorsement of any of these entities by the Virginia Department of Education, the Virginia Board of Education, or the Commonwealth of Virginia.