Key factors and challenges of research-informed policy making in ECEC: examples from longitudinal studies

Abstract

In this commentary, we draw on examples from the thematic issue ‘Longitudinal studies in ECEC—challenges of translating research results into policy actions’ to discuss how research from longitudinal studies in ECEC can potentially inform policy making. Thereby, we point out the challenges in the relationship between research and policy and suggest three key factors that can promote the impact of meaningful research in policy decisions. The first factor comprises aligning research questions with policy priorities as well as research and policy making processes. In an attempt to systematise existing forms of research and policy interaction, we present ideal–typical models of collaboration and relate them to studies in this thematic issue. Second, we argue that research used in policy decisions should rely on sound methodology, including state of the art techniques of sampling as well as reliable and valid measures. While acknowledging that various research designs are required to get a bigger picture of the complexity of ECEC, we call attention to the particular potential of longitudinal studies to inform policy making. Third, we stress the importance of disseminating research results to different stakeholders and outline challenges of translating research findings into policy decisions. Finally, in the outlook, we discuss how collaboration between ECEC research and policy making can be increased and which barriers need to be overcome.

Background

The importance of good early childhood education and care (ECEC) for children’s development and learning is well documented (Anders et al. 2013; Burchinal et al. 2010; Larazzi and Vandenbroeck 2012). Not surprisingly, ECEC systems are exposed to high expectations by many different stakeholders, for example participation in ECEC is expected to reduce educational inequality between girls and boys from diverse backgrounds (e.g. social, linguistic and cultural), better prepare children for further education as well as increase the compatibility of family and work life.

Policy makers who are responsible for steering ECEC systems face the challenge that ECEC systems are constantly in flux. Due to a rising number of children with diverse backgrounds in ECEC settings, higher demands for places and longer opening hours, the lack of qualified staff, changes in family constellations, working environments and educational systems, as well as the society as a whole constant adjustments are required (Tietze and Eckhardt 2013). What role can research play in the design of ECEC policies? A substantial amount of literature discusses the relation of research and policy making in the field of education. Some prefer research to fulfil the function of offering new ways of thinking and of changing our understanding of certain phenomena (often referred to as the enlightenment function of research), others view educational research in a more technical sense (often referred to as the engineering function of research). The latter entails that researchers are requested to collect data and identify cause-and-effect relations to solve specific problems (cf. Hargreaves 2006). We argue that research needs to fulfil both functions, a more ‘fundamental’ or enlightening one by raising new questions, developing theories, exploring new methods and spreading these advances into every-day, professional and policy discourses, as well as a more ‘applied’ or engineering one, where the aim is to identify causal mechanisms and to find concrete solutions to existing problems. To give an example of the enlightenment function in the field of ECEC, research can contribute to explore and identify new dimensions of ‘good’ ECEC. As for the engineering function, research on factors that positively impact children’s learning and well-being as well as ECEC professionals’ job satisfaction can potentially support policy makers to make informed decisions, so that ECEC settings become fruitful learning and well-being environments for children, worthwhile working environments for ECEC professionals as well as help families to better combine family and work commitments. This kind of research might also offer policy makers a basis to justify spending on certain areas of ECEC (e.g. improving staff-to-child ratios). To justify the total budget spent on ECEC, politicians might also draw on longitudinal studies that have explored the economic returns on investment of ECEC to society [see for example Schweinhart (2016) on US studies like the HighScope Perry Preschool Study, the Abecedarian Child Care Study and the Chicago Longitudinal Study]. Within the engineering function, research can produce different types of knowledge: descriptive knowledge that describes and summarises observed features of an object with the help of tables and indicators, explanatory knowledge that informs about assumed causal relationships and operative knowledge that informs about the impact of determinants that can be modified by policy (Kalicki et al. forthcoming). Although both functions of research—enlightenment and engineering—are equally important, in this commentary, we focus more strongly on the engineering function. Importantly, and as will be pointed out repeatedly in the following, the engineering function draws on theoretical and methodological advances in more fundamental research.

Policy makers’ key instrument to shape ECEC systems is legislation, for example they are able to increase access to ECEC for specific groups of children, enhance funding for the training of ECEC staff (e.g. Melhuish 2016) or grant child care subsidies aimed at reducing the financial burden of parents’ ECEC costs (e.g. Kim et al. forthcoming). Further, policy makers are able to provide targeted funding for the implementation of specific pedagogical approaches [see Anders et al. (2016) on the implementation of language education embedded into every-day pedagogical routines] or prevention programmes (as advocated by Lee et al. forthcoming).

In the effort to base political decision-making and funding processes regarding the ECEC field on solid ‘evidence’, policy makers are increasingly funding large-scale longitudinal (cohort) studies. Compared to cross-sectional studies, longitudinal studies are able to include multiple measurement points and not only capture correlations but potentially also inform about causal linkages between different aspects of ECEC and child development. Often a structure–process–outcome model is applied. In this case, analyses are aimed at identifying relations between structural aspects of ECEC (structural quality), the observable pedagogical quality in ECEC (process quality), and child development (outcome quality) (cf. NICHD Early Child Care Research Network 2002). Assessing structural, process and outcome quality longitudinally is very complex and costly, thus not all longitudinal studies assess all aspects of the structure-process-outcome model directly but provide proxies. Also, in order to make causal analyses of the effects of ECEC on children’s development, it is essential to assess characteristics of the family background and ideally the home learning environment, as alternative influencing factors. Information about potential effects of certain factors is referred to as explanatory knowledge, and may be considered helpful for policy decisions in the field of ECEC (e.g. Bromme et al. 2014; Kalicki et al. forthcoming).

The health sector is often referred to as a model for research-informed policy in the field of (early) education. Compared to the health sector, systematic large-scale and longitudinal investigations of the relations between structural ECEC characteristics and pedagogical or educational processes on children’s learning and development are a fairly new development (see Anders 2013 for an overview). The studies discussed in this issue show that some countries have a longer history of longitudinal ECEC studies (e.g. in the US, Head Start was launched in 1965 and in the UK, the ‘Effective Pre-school Primary Education Study—EPPE’ in 1997), whereas in other countries, sophisticated research in this field was initiated more recently (e.g. in Germany, ‘Educational Processes, Competence Development and Selection Decisions at Preschool and School Age-BiKS’ was launched in 2005 which was the precursor study of the ‘National Educational Panel Study-NEPS’ launched in 2008, and in Korea, ‘the Panel Study of Korean Children-PSKC’ was launched in 2006, and New Zealand’s ‘Growing Up in New Zealand Study’ in 2009).

Drawing on research evidence for making policy decisions is not trivial. It includes risks and has strong limitations. One major limitation is, for example, that research evidence is not genuine truth in itself, but fallible, preliminary and, often, contradictory (Hammersley 2013). Each research study involves operationalising constructs as well as generalising and interpreting results which always comprise the possibility of errors and misinterpretations. Even by using very elaborate research, it is not possible to estimate all consequences that a policy measure may have. Therefore, policy decisions cannot be solely based on research but are additionally driven by ethical, pragmatic or strategic considerations.

Furthermore, the analogy to the health sector regarding research-informed policy has its limitations because of important differences between medicine and (early) education. For example, in the health sector, systematic reviews of studies that are based on randomised trials are considered the most robust evidence. In (early) educational research, which is primarily based on research from social sciences, experimental designs with randomised trials are more difficult to realise and it is hardly ever possible to control all influences. Study results, therefore, depend on various context conditions and are less generalisable than results from experiments conducted in more technical sciences. Thus, systematic reviews in (early) education research involve a higher degree of evaluating and interpreting findings with regard to the context of the study (Clegg 2005). This demonstrates that in the field of ECEC, research studies cannot easily ‘compete’ with studies conducted in other fields like the health sector, e.g. concerning certain standards of evidence (e.g. randomization). Research with children involves an ethical obligation to not deny children access to positive experiences or ‘good’ ECEC as well as to include them as study participants in their natural contexts (i.e. together with the peers and caregivers they are attached to). These natural contexts are socially and culturally diverse and unpredictable (Vandenbroek 2012).

Acknowledging the limitations of research, this commentary aims to discuss three key factors that can make research which is aimed at fulfilling the engineering function count in ECEC policy making at the macro-level¹: Alignment of research questions with policy priorities, use of Sound Methodology and Dissemination of results in order to promote their information potential for policy actions (adapted from Taggart 2010).

Alignment

Alignment² in our opinion covers two aspects: (a) how aligned are research questions with policy priorities and (b) how aligned are research and policy making processes which enable opportunities for collaboration between the two groups.

Alignment of research questions with policy priorities

Large-scale longitudinal (cohort) studies in ECEC with a wider focus enable researchers to explore new dimensions in ECEC as well as answer pressing questions of policy makers who might be looking for guidance when developing new structural policies (e.g. legal entitlement) or promoting certain pedagogical programmes. The articles in this issue demonstrate how country-specific developments motivated governments to fund large-scale longitudinal studies. Weinert and colleagues (2016), for example, describe that the newborn cohort was first only intended as an add-on project to the National Educational Panel Study (NEPS), but in the end, fully integrated into the panel study to answer questions arising in connection with new developments in the German ECEC sector: the extension of legal entitlement to 1 and 2 year olds from the 1st August 2013 and the parallel expansion of ECEC services for this age group [already legally consolidated in the Childcare Funding Act (KiföG) in 2008]. Against the background of the growing participation of under 3 year olds in Germany, Weinert and colleagues (2016) chose to focus on exploring early roots of social disparities in very young children. The authors Lee and colleagues (forthcoming) and Kim and colleagues (forthcoming) also show how they orientated their research questions on new developments in the ECEC field in Korea. For example, in the light of the recently introduced national Nuri curriculum, which among other aspects emphasises the support of children’s social skills (‘character education’), Lee and colleagues (forthcoming) conducted a latent profile analysis to explore different dimensions of children’s social behaviour. They provide an interpretation of their findings addressed at policy makers with regard to how children’s social development can be better supported in Korean ECEC settings. Kim and colleagues’ (forthcoming) research question was motivated by the declining fertility rate in Korea, and they investigated whether the governmental ECEC benefit of KWR 100,000 influenced couples’ childbirth decision.

A challenge of research which is closely linked to policy is that funding and policy makers’ general commitment to research studies can be bound to the legislative periods of governments and, thus, not be reliable in the long term. However, government changes do not necessarily imply that studies are abandoned. Melhuish (2016), for example, describes how the initial idea for EPPE (later extended to secondary school and renamed to ‘Effective Pre-school, Primary and Secondary Education Study-EPPSE’) came in 1996 in connection with the conservative government’s aim to evaluate the voucher scheme for nursery places for 3–4 year olds. Although the new Labour government abandoned the voucher scheme in 1997, an increasing political interest in ECEC resulted in additional funding for the EPPE study. The Labour government aimed to base their social reforms on research evidence and therefore had a high interest in the EPPE study. The goal was to transform the wide variety of different ECEC services into a more homogeneous system through policy initiatives and to find out what impact ECEC has on children’s development (Taggart et al. 2008). Hence, the EPPE study was suggested in a time when policy makers were in need of research evidence.

Similar to the EPPE study, Growing Up in New Zealand was closely related to political needs. The study evolved from a call for proposals for longitudinal studies from the New Zealand government in 2004. The goal of the study is to ‘provide a robust, relevant evidence base to inform policy related to children and their families in twenty first century New Zealand’ (Bird et al. 2016). The authors Bird and colleagues (2016), who represent the research side, indicated that the study was developed in close collaboration with the government. The data collection strategy is aligned to relevant policy questions and aims to better understand disparities, raise achievement and enhance equity.

Next to longitudinal (cohort) studies exploring the interrelations between different aspects of ECEC in a broader sense, longitudinal programme evaluations are examples of how specific policy interests can explicitly direct research questions [e.g. the evaluation of the highly controlled intervention HighScope Perry Preschool Study, Schweinhart (2016), or the longitudinal evaluation study of an early education programme ‘Core daycare centres language & integration’ in Germany, Anders et al. (2016)]. Programme evaluations are user-oriented in that they aim at informing policy makers whether necessary adjustments to the programme need to be made (e.g. Böttcher et al. 2009). They are also intended to provide policy makers with timely evidence whether public money has been invested wisely. Anders and colleagues, for example, describe how they were commissioned by the Federal Ministry of Family Affairs, Senior Citizens, Women and Youth (BMFSFJ) to specifically evaluate the newly introduced programme which focused on early language education embedded into daily routines. Key questions were as follows: how does the programme influence the work of the specifically trained language experts and the work of the professional teams as a whole? How does the programme affect children’s language development—in particular, children learning German as a second (or third or more) language—and their families?

Alignment of research and policy processes: models of collaboration

Evidently, policy makers have the power to introduce change and have experience of what is feasible and justifiable to implement. However, they may not always be experts in the field. Researchers on the other hand have expert knowledge, but they cannot make decisions (Boyd 2013). Therefore close collaboration between the two groups is essential. Collaboration between policy makers and researchers can be described with different models. Traditionally, researchers conduct their studies independently and policy makers use the findings for policy making—in this model, researchers and policy makers are considered as two separate communities (Furlong and Oancea 2006) and is referred to as a one-way linear transfer model (Sylva et al. 2007). In a two-way model, policy makers are interested in certain questions and they commission a research team to conduct research on these questions. Even though policy makers are involved in the research right from the beginning, the model can be characterised through knowledge transfer rather than knowledge exchange. The third model, a more recent model, can be described as a partnership model and includes knowledge exchange. Within the partnership model, both groups, researchers and policy makers, are involved in shaping and implementing the study and disseminating the findings. Research in the health field showed that the one-way and two-way models did not lead to actual change in practice, whereas the partnership model did (cf. Sylva et al. 2007).

Each of the three collaboration models has its advantages and disadvantages. The one-way linear transfer model provides researchers with the freedom to ask fundamental questions; there is no pressure to serve a certain institution or group of people or offer concrete practical solutions. However, this can mean that researchers might be exploring questions that are not always aligned to urgent information needs of policy makers. The results will likely not be used to introduce immediate change to the ECEC sector but serve to learn more what exactly happens when ECEC professionals or parents are interacting with their children [see for example Weinert et al. (2016) on their observation of mother–child interactions]. The two-way model can be seen as target-oriented since it addresses pressing policy questions and increases the probability that change will be introduced. However, there is the danger that ‘only’ commissioned research is conducted and the necessary exploration of research questions that are not straightforwardly policy relevant is neglected. The partnership model seems to be the most adequate model to introduce change, but close collaboration is time-consuming and can be tricky due to power differentials or the fact that the two groups hold different mandates.

Intertwined with the quality of collaboration (i.e. the direction of knowledge transfer) is the quantity of collaboration. In the attempt to embed typical phases of research in an ideal–typical policy cycle (see Fig. 1), main opportunities for collaboration might be during the first phase, when either researchers or political or societal stakeholders identify a ‘problem’ (e.g. What can we do to tackle the shortage of ECEC professionals?); in a second phase, based on prior research if available, the policy or intervention (e.g. higher wages as an incentive to attract people to become ECEC professionals) is formulated independently by policy makers or in collaboration with researchers; in a third phase, the policy or specific programme is implemented; and in a fourth phase, either researchers are commissioned to evaluate the policy or researchers might independently choose to follow up a research question in connection with the current policy (e.g. Do higher wages attract more people to become ECEC professionals?). The research process typically includes formulating the research questions and hypotheses, developing the study design, collecting and analysing data. Following the analyses, conclusions can be drawn and researchers might be able to formulate policy recommendations. At this point, policy makers decide whether to continue, adapt or terminate a policy or programme based on the recommendations of researchers or their own interpretations of research findings. If the ‘problem’ or knowledge gap that was identified in the first phase can be eliminated, the next step would be to either identify a new ‘problem’ or refine an earlier question (e.g. In spite of higher wages, there is a shortage of ECEC professionals. What else can we do?). With this step, the cycle recurs. Of course, this policy-research-cycle below needs to be considered as a heuristic orientation frame, and in reality the described phases tend to overlap or take place in parallel (Blum and Schubert 2011).

Fig. 1

Policy-research-cycle; based on the typical phases of the policy cycle developed by Jann and Wegrich (2009) (taken from Blum and Schubert 2011, p. 105) and the typical phases in the process of research (taken from Frankfort-Nachmias and Nachmias 1992, p. 22); own translation and adaptations

The articles in this issue report that all the respective studies are in some way linked or aligned to policy making. Also, ‘real’ collaborative processes—as described in this issue—seem to include characteristics of more than one model, and quality and quantity of collaboration differ over the time course and different stages within a single study. However, the models are helpful as an analytical tool in order to describe differences and commonalities and to point out particularities of collaboration within a certain study. Growing Up in New Zealand and EPPSE perhaps come closest to a partnership model: Growing Up in New Zealand seemed to have successfully established an exchange platform, the so-called Policy Forum that allowed researchers and policy makers to frequently exchange pressing issues and match policy priorities with data analyses indicating a collaborative partnership (Bird et al. 2016). Similarly, collaboration between researchers and policy makers within the EPPSE context seems unique due to their intense and formalised collaboration (e.g. principal investigators met monthly with policy makers discussing a mutually agreed agenda; Sylva et al. 2007).

Anders and colleagues (2016) describe how collaboration moved from a two-way model (commissioned evaluation) to a partnership model. Findings were continuously reported back to the governance board and steering group. This made it possible to adjust and improve the policy measures along the way. For example, the first survey results showed that the language experts were experiencing difficulties with regard to passing on their newly gained expertise to all team members in the setting; they were mainly engaged in working with children individually. In collaboration with the ECEC providers, it was possible to increase in-house trainings for the language-related quality process (Anders et al. 2016).

These are only some examples that show how collaboration occurs in real studies. For future research, it would be interesting to tackle the question empirically: what are consequences of the different types of collaboration for both policy and research in terms of outcomes (i.e. quality of policies, quality of research and unintended consequences)? Although these outcomes are extremely difficult to measure, it might be worth investigating this relationship in order to get closer to an answer on an optimal degree of alignment between research aimed at fulfilling an engineering function and policy making.

Sound methodology

Studies that are not carefully set up regarding methodology risk producing biased results. Therefore, findings that are used to inform policy decisions should always be based on sound methodology (Taggart 2010, p. 207). Methodology comprises the design of a study (sampling and measures) and the statistical methods of analysis that are applied. How can policy makers make sure that the studies they consult are trustworthy? One important issue in order to be able to draw conclusions based on the results is a sample that adequately represents the population under study. Regarding ECEC, sampling strategies, however, often involve more complexity. First, complete lists of ECEC centres or enrolled children often are not readily available. Second, participation of selected units can often not be realised in a random way, for example because specific ECEC centres or children and their parents are not willing to participate. This might result in a systematic bias of the sample. Also, a sample often has to represent different levels (e.g. regions, ECEC providers, ECEC settings, groups and children, e.g. Anders et al. 2016; Bird et al. 2016; Weinert et al. 2016) and make sure that the relevant subgroups (e.g. ethnic groups, e.g. Bird et al. 2016; Weinert et al. 2016) are large enough to draw comparisons to other subgroups. The quality of the sample can be assessed by looking closely at the description of the sample selection. A good indicator for a trustworthy sample is that researchers of a study argue plausibly that the sample under study does not differ from the population (in population studies), respectively, that treatment and control groups do not differ in characteristics other than the treatment (in randomised controlled trials and evaluation studies).

The quality of the results and whether policy makers take them into account also depend on the validity and reliability of the measures used. As Anders and colleagues (2016) describe, it is a great challenge to measure the development of very young children since existing measures only cover some domains and often are designed for younger children or other cultural contexts. The researchers point out that the employed battery of validated sub-scales initially developed to assess pedagogical processes in the US context were not fully able to capture the special nature of interactions that occur when the pedagogical approach of embedded language education is applied. Yet, the urgent need for timely results did not allow the researchers to pilot and validate the adapted observational tool for the German ECEC context. According to Anders and colleagues (2016), it therefore remains an open question whether rather low correlations between two measurement points (ranging between 0.11 and 0.18) show that the language-related quality in the observed settings tends to be very unpredictable or that relevant aspects of the pedagogical approach might not have been adequately captured by the observational tool, implying that the tool might need to be improved. Therefore, besides aiming at the alignment of research questions and current policy needs, room for innovative explorations is essential. ‘Knowledge gaps’ are not always evident from the start but emerge in the process of researching. Thus, on the one hand, there is a strong need for innovative measures and on the other hand, developing new instruments requires careful checking for validity and reliability in elaborate and costly pilot and validation studies.

In this issue, several authors describe the development of new measures. Weinert and colleagues (2016), for example, describe a newly developed indicator for basic information processing in young children (visual attention and speed of habituation) and mother–child interaction and provide information on the validity and reliability of their measures. Melhuish (2016), as another example, reports on the development of the SSTEW scale (Sustained Shared Thinking and Emotional Well-being) to assess quality in ECEC. The scale focuses on adult–child interactions by measuring sustained shared thinking strategies and ECEC professionals’ support of self-regulation abilities in children.

The studies described in this issue are all longitudinal studies, i.e. studies that follow individuals over at least two measurement points, most of them for several years, some of them even for decades. Longitudinal studies (based on a sample of a population or intervention studies with random assignment to treatment and control groups) are not the only way to generate robust findings that might inform policy making. On the contrary, it is important that also other types of studies, such as indicator-based monitoring and reporting or large-scale cross-sectional studies, are carried out, because each of these studies creates a different type of knowledge (cf. Kalicki et al. forthcoming). Also, qualitative studies can provide valuable insights although the scope of their results is sometimes more limited than that of quantitative large-scale studies. Furthermore, compared to cross-sectional studies, longitudinal studies have several strengths regarding the interpretation of causal relationships (Kalicki et al. forthcoming), the range of research questions that can be answered and the usage of advanced statistical methods as described in more detail in the following.

How longitudinal studies may provide indications of causal relationships

Politicians in the field of ECEC need to be informed on causal relationships (e.g. between ECEC characteristics, on the one hand, and children’s development and learning, on the other), so that policy making can benefit from research. Causality is a concept that is broadly discussed in philosophical and statistical literature. It is generally acknowledged that observing and testing causal relationships directly are very difficult, if not impossible. However, it is possible to find good indications that a correlation might indeed be a causal relationship. Longitudinal data are considered to have several advantages compared to cross-sectional data as a source for such indications (for a more detailed description see Blossfeld et al. 2009; see also Weinert et al. 2016). For example, longitudinal data include temporal information by default and, thus, allow ordering of earlier and later events, and cross-sectional studies only exceptionally do so (for example when they collect retrospective information or traits that do not change, Wunsch et al. 2010). Also, longitudinal data allow specifying the time that elapses before an effect emerges, whereas in cross-sectional studies, the time interval is left unspecified. Hence, longitudinal data provide relevant information for the timing of policies and allow disentangling temporal aspects such as age, dosage and duration. Third, also the explanatory variables (determinants) may change in time. For example, when analysing the effect of ECEC characteristics on children’s development of language and literacy skills, it is important to take into account if and when a child changes ECEC settings. Longitudinal data consider such information, whereas cross-sectional data usually only include the ECEC setting at the time of the interview or/and assessment. Fourth, claiming that an effect (e.g. differences in children’s levels of skills) can be traced back to a cause (e.g. exposure to ECEC) requires ruling out other possible influences (e.g. family background). This is a major challenge. Since longitudinal data provide the baseline and the development of characteristics at the individual level, they provide better options to control for selection effects than cross-sectional data. For example, when analysing the impact of early language education embedded into daily preschool routines on children’s language skills, the evaluation study described in Anders and colleagues (2016) compares children’s development of language skills of those exposed to the programme to those not exposed to it. Comparing how children’s language abilities have progressed, and not just comparing their attained levels, reduces the risk of assigning a causal effect to the programme that really is only the result of selection into the programme.

Measuring change and stability in longitudinal studies

Longitudinal studies are designed to measure change and stability over time. This is a particularly useful feature of research on early childhood because children develop fast and at different paces. Patterns of development are therefore more interesting than a snapshot of children’s skills or behaviour at a certain point in time. For example, as Lee and colleagues (forthcoming) demonstrate, it is quite usual that behavioural problems change between ages 4, 5 and 6. In order to distinguish problematic and unproblematic developments, it is more revealing to analyse whether behavioural problems decrease, increase or are stable than only to look at the level of behaviour problems at a certain age.

Not only individual development is crucial in ECEC but also changes in institutional contexts (e.g. types of ECEC, availability of places in ECEC, access to free ECEC compared to high parental fees for ECEC) and historical contexts (e.g. women’s increasing participation in the labour market). In England, for example, free ECEC for 2-year-old children in the 40% most deprived families was extended in 2016/17 from 15 h a week to 30 h a week. SEED, which started in 2014 with 2 year olds, can be used to investigate how this extension affects individual attendance of children from lower-income families in ECEC (Melhuish 2016).

Measuring path dependency and long-term effects in longitudinal studies

A particularly relevant topic for ECEC policy making is long-term effects of ECEC. In the US, longitudinal studies that started in the 60s and 70s (e.g. Abecedarian Child Care Study, Chicago Longitudinal Study and, HighScope Perry Preschool Study; Schweinhart 2016) showed strong return on investment of high-quality ECEC for children from economically deprived families. Children who attained these programmes showed higher school achievement, higher high school graduation rate, higher employment rate, higher earnings and, regarding two of the programs, a lower crime rate (Schweinhart 2016). EPPSE results show that preschool effectiveness still affects literacy and numeracy at ages 11 and 16 (Melhuish 2016). Tracing back a long-term effect to the causal influence of certain ECEC characteristics, of course, requires caution because there are many other possible causes of which only a few can be observed and controlled while many other factors remain unobserved and even unknown. Therefore, causal relationships cannot be proven with certainty. However, with longitudinal data, possible alternative causes can be better controlled since they include baseline measurements and the development of characteristics is assessed (as described above).

Documenting transitions in longitudinal studies

For policy makers to initiate targeted support measures, it is crucial to know which groups have difficulties dealing with a transition and why (Dunlop and Fabian 2007; Lillejord et al. 2015). Longitudinal studies are particularly suitable to provide such information because they have data on the same individuals before and after a transition. Studies described in this issue (e.g. EPPSE, Growing Up in New Zealand, NEPS and SEED) collect information before and after crucial transitions such as changing between ECEC institutions and starting to go to school. Growing Up in New Zealand and NEPS start even earlier (in pregnancy/6 months after birth), so that the transitions to first external care can be analysed.

Longitudinal studies also have some disadvantages that have to be taken into account (for a more detailed description see Wunsch et al. 2010, pp. 9–12). Probably the most important one is panel attrition. As already described above, sampling strategies in ECEC can be quite challenging. In longitudinal studies, additional selection bias results from attrition in panel studies and selective reporting in retrospective studies. This may reduce the representativeness of data.³ Other disadvantages are that data editing and analysis can take longer than in cross-sectional studies, because data handling is more complicated and that measuring the same characteristics at different points in time is very challenging for the validity and reliability of measures. These costs, however, have to be considered against the background of a much higher research potential as described above.

Studies that inform policy decisions should be based on sound methodology. This includes representative samples, reliable and valid measures, and the diligent dealing with statistical methods as well as limitations of design, methods and analyses. Since we focus on longitudinal studies in this thematic series, we described particular assets that longitudinal data can potentially offer to research-informed policy making as well as the research limitations that have to be considered.

Dissemination

Whereas alignment and sound methodology are usually addressed before or during the implementation of the research study, the communication of research findings and their potential translation into policy happens after (first) analyses have been conducted. So, in a first step, the research findings can be spread to different stakeholders, and in a second step, policy makers can actually use the findings to introduce or adapt legislation and funding to make improvements in the ECEC systems.

Addressing different stakeholders

A very common strategy in the research field to spread research findings is to publish research findings in scientific journals and present results on scientific conferences. This is a strategy to address other researchers in the field and does not often include policy makers. Furthermore, it is important to be cautious about research findings. Scientific journals are more likely to publish statistically significant research findings and authors are also more likely to submit papers if their results reach a significant level. Findings that do not reach a level of statistical significance might still bear valuable information, but access to these findings is more difficult since the probability to find these published is lower (Franco et al. 2014). To address policy makers, researchers have to convey the information through other channels. A variety of different strategies can be applied to spread research findings to policy makers. Melhuish (2016) describes that in addition to publications in academic journals, EPPSE researchers regularly presented their research to the media and participated in discussions and debates. Furthermore, he describes that research findings were presented in “clear and firm policy conclusions and recommendations” (Melhuish 2016). The formulation of such policy recommendations, however, is often not straightforward as there is always interpretation involved. It is important to clearly indicate what are the pure results and their limitations and what are the conclusions derived from the results and why are alternative conclusions less plausible. According to Schweinhart (2016), findings of the HighScope Perry Preschool Project in the United States were published in academic journals first and then communicated to policy makers and the public. These communication approaches are highly effective but it needs to be considered that it takes additional time for researchers who are mainly engaged in fundamental research like developing new assessment tools to write policy briefs, talk to the media or present their findings to the public. As mentioned previously, Bird and colleagues (2016) describe a close collaboration between policy makers and researchers throughout their study through the establishment of a Policy Forum. For the Policy Forum, broad reports are produced to communicate important research findings. Additionally, specific policy briefs are written to ensure that policy questions are answered and if senior policy makers are interested in specific analyses, they can request specific policy briefs in addition to the broad reports.

How effectively results are translated and used for research-based policy development highly depends on how clearly and easily accessible results are presented and whether policy makers trust the findings. Importantly, methodological limitations need to be discussed explicitly so that policy makers can decide whether the results can serve as an ‘evidence-base’ for new policy measures (Bromme et al. 2014). However, this is a great challenge for both sides, for researchers to represent the methodological limitations in a way that they are understandable to non-experts of the applied methods and for politicians to take the time and make the effort to understand the complex methodological issues.

However, to tackle this challenge of treating findings and limitations adequately in the first place, some sort of communication has to be established between researchers and policy-makers. A meta-analysis by Innvaer, Vist, Trommald and Oxman (Innvaer et al. 2002), which reviewed 24 research studies that included interviews with health policy decision makers, showed that personal contact between researchers and policy makers was one of the most important facilitators of the use of research evidence. Another study (Ross et al. 2003) interviewed researchers and health system managers who collaborated with one another. This study indicated that the benefits of the collaboration (i.e. research became more applicable for users and researchers gained a ‘big picture’) outweighed the costs (i.e. time and effort). However, it needs to be taken into account that the findings above mainly concern the field of health rather than ECEC. Furthermore, referring back to the paragraph ‘Alignment’, proper alignment is necessary for a successful translation of research findings.

Translation into policies

According to authors in this issue, research has been used to inform policy actions in various ways: free part-time ECEC for children aged 3 to school-entry (came into effect in 2004) based on EPPSE findings (Melhuish 2016); extended role-out of the embedded language education approach combined with a system for professional support for the language experts in a new government-funded programme called ‘Language preschools (Sprach-Kitas)’ starting in 2016 (Anders et al. 2016); more recent findings will still show whether and how they will manifest themselves in policy actions (Lee et al. forthcoming; Weinert et al. 2016). However, authors in this issue also point out that some policy actions were not solely supported by the research evidence but influenced by other factors such as public preferences. EPPSE and a second longitudinal stage of the NESS impact study indeed revealed that the so-called integrated children’s centres (Sure Start)—centres offering a combination of childcare, education and various family support services—seemed to have a particularly beneficial effect on children’s prosocial behaviour, self-regulation as well on parent–child relationships (etc.). However, high popularity of Sure Start amongst parents equally motivated the government’s decision to increase the number of Sure Start programmes operating the children’s centres model by 2010.

Outlook: How can research and policy interact in the future to create more positive experiences for children and ECEC professionals?

Undoubtedly, policy making can be a ‘messy’ business. Policy measures are never only informed by research but need to take into account various factors, such as public opinion, social developments, values, time pressure or cost limitations (Boyd 2013; Davies 2004; Hargreaves 2006). Furthermore, research is not always able to capture the full complexity and diversity in ECEC. This also applies to the longitudinal studies in this Thematic Series. Nevertheless, they can provide valuable information for ECEC policy making and this commentary attempted to explore how the impact of meaningful research might be increased. Thereby, three factors were highlighted—alignment of research questions with the ongoing political agendas in combination with sound methodology and a systematic and continuous dissemination strategy (adapted from Taggart 2010). Most of the described studies employ strategies to ensure that their findings are acknowledged by policy makers. A frequently practised strategy is to exchange knowledge and interests and discuss results during regular meetings between the research teams and policy makers. However, looking at the field of ECEC as a whole, these are scattered efforts of single studies rather than common strategies. What challenges have to be overcome in order to link both fields more closely and make research count in ECEC policy making? The alignment of research questions with policy questions seems to be hindered due to the lack of joint panels in which researchers and policy makers regularly exchange knowledge and ideas. Perhaps a partnership model (cf. Sylva et al. 2007) can promote this exchange. Close collaboration, however, involves the challenge to guarantee that the research process (e.g. decisions on methodology and the analysis of data) is carried out without political influence. Moreover, it has to be ensured that fundamental research that is not necessarily directly linked to pressing policy questions can be realised and receives enough funding. One solution was suggested by Schweinhart (2016) who argues that a certain amount of funding—maybe 5%—should always be allocated to fundamental research and exploration within large-scale longitudinal studies. This point is also strongly linked to methodology. Various authors describe how they struggled with the lack of instruments to adequately assess pedagogical interactions (Anders et al. 2016; Melhuish 2016) or young children’s development beyond ‘easy’ measurable domains like literacy or early numeracy (e.g. Weinert et al. 2016) and might have needed more time and money to conduct thorough pilot or validation studies. This stresses the importance of research that is detached from direct use for policy making—in this case targeted at the further development of methodology. It seems that longitudinal studies in ECEC need to achieve a balance between offering new perspectives on a topic as well as providing information on potential effects of policy measures or pedagogical practices. Further, the complexity of methodology creates barriers to collaboration. Although policy makers have a significant amount of knowledge about an issue in which policy is to be made, they may not always have the time to fully grasp all details of the methodology. On the researcher’s side, it is a great challenge to describe methods and their implications on the findings in a simple, however precise way. This requires transparency regarding all methodological details, like non-response, panel attrition, instrument description or independency of external evaluation team and the measure under investigation (e.g. Egert and Hopf 2016). Further, researchers should not hesitate to openly communicate that conclusions can only be drawn to certain degrees and applied methodologies are continuously improved (Hargreaves 2006). Communicating research findings to a non-scientific audience requires more comprehensive forms of communication, and might not be rewarded by the scientific community. Again, incentives for researchers to invest time in a non-scientific audience are scarce. Of course, there are examples of solutions to meet these challenges. Several approaches are demonstrated in the studies described in this issue, where time and funding are specifically dedicated to the task of disseminating and translating findings into politics and individual researchers have taken great efforts to communicate their findings to policy makers (e.g. the Policy Forum, Bird et al. 2016). There are also efforts to systematically review the state of research regarding policy-relevant questions that take into account several studies. Following the example of the Cochrane Collaboration in the health sector, the Campbell Collaboration initiates systematic reviews of research on the effectiveness of social interventions including the field of early childhood. Furthermore, specialised online-platforms are a promising attempt to summarise research and make it readily available to policy makers and practitioners. For example, the Websites ‘What Works Clearinghouse’, ‘Doing What Works’, or ‘Best Evidence Encyclopedia’ provide easily comprehensible research reviews in education, each with a special section on early childhood. Another example is the project ‘CLOSER’ (Cohort and Longitudinal Studies Enhancement Resources) that aims at promoting use, value and impact of eight longitudinal studies in the UK. Moreover, there are think tanks and journals that fill the gap between research and policy by taking over the task of translation. At the international level, the Thematic Working Group on Early Childhood Education and Care of the European Commission (2012–2014) or the OECD Network on Early Childhood Education and Care, for example, also serve(d) as platforms for knowledge exchange; empirical findings from ECEC are presented and discussed and should stimulate policy making in the member states. However, the OECD itself is critiqued that they use their own empirical data, e.g. collected by PISA (Programme for International Student Assessment) or by their new International ECEC Staff Survey (currently in the pilot phase) as instruments of governance with which they not ‘only’ advise countries but aim to strengthen their own influence on international politics (e.g. Bloem 2016).

The great challenge of the next decades in ECEC policy and research, however, is to connect both fields in a more systematic way so that meaningful research findings are available to policy makers and find their way into policy decisions.