http://orcid.org/0000-0002-4739-7151
Figures
Abstract
Background
A physical education (PE) intervention for young refugees was designed combining physical activity within the context of primary PE games with second language learning activities in German. The intervention was based on theoretical implications from the field of second language acquisition and evidence for positive effects of physical activity on cognitive outcomes (e.g. language acquisition). The aim of this study was to analyze short term effects on second language acquisition.
Methods
Sixty-one young refugees were included in the study (age: 8.5 ± 1.4 years). The intervention group participated in language-enriched PE lessons based on an elaborated approach to second language learning acquisition. The control group did not receive any treatment. Both groups were pre-and post-tested in domain specific vocabulary, listening comprehension and use of local prepositions within the context of primary PE games.
Results
Results from linear mixed-effect modelling suggest that the intervention group significantly improved domain specific vocabulary and listening comprehension in comparison to the control group.
Conclusions
The intervention was successful since the PE lessons contributed to the second language acquisition of young refugees. Therefore, this learning approach might also be useful for physical activity based second language learning activities in other PE contexts for early second language learners in primary school.
Citation: Krüger M (2018) Second language acquisition effects of a primary physical education intervention: A pilot study with young refugees. PLoS ONE 13(9): e0203664. https://doi.org/10.1371/journal.pone.0203664
Editor: Hanna Marno, Central European University, HUNGARY
Received: October 1, 2017; Accepted: August 25, 2018; Published: September 17, 2018
Copyright: © 2018 Mirko Krüger. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: We acknowledge support by the Open Access Publication Fund of the University of Duisburg-Essen.
Competing interests: The author has declared that no competing interests exist.
Introduction
The number of refugees who, due to conflicts or persecution, left their home country fleeing to other countries is at the highest level in recorded history [1]. Many of them are children who are still required to attend school [2]. This increase of refugees also affects Germany and the whole German school system [3]. Since 2015 many German schools face the challenge to integrate these new students in everyday school routines to meet their academic needs as well as their wellbeing needs. Among others (e.g. enhancing students´ general knowledge), teachers are asked to foster refugee children’s academic performance, especially their German language proficiency (L2). Therefore, specific L2 learning activities are implemented into regular school routines. That is, individual language courses implemented additional to regular classes and integrated language learning [4], implemented within regular classes, even within Physical Education (PE) classes.
To date, knowledge on the integration of L2 learning is scarce in German literature on PE [5] and only few methodological approaches for PE and L2 language integrated learning (intPE-L2) exist [5,6]. All of them consider the promotion of basic interpersonal communication skills (BICS) and cognitive academic language proficiency (CALP) [7] as overall learning goals incorporating several learning strategies from the field of L2 learning literature [8]. However, these intPE-L2 approaches have not yet been examined regarding their effectiveness within school-based interventions. Of note, intervention studies from early childhood education showed significant improvements of children’s oral and written language acquisition after integrating PE and L2 language learning activities [9,10]. Similar effects have also been found for preschoolers with speech impediment [11]. Furthermore, there is a plethora of evidence indicating that increased physical activity (PA) leads to better academic achievement [12–15], which is also evident for different language learning skills [16–17].
The theory of brain-based learning is suitable to explain these effects. The theory states that moderate to vigorous physical activity (MVPA) effectively stimulates the brain [18]. In short-term, PA leads to immediate biochemical changes in cerebral regions thereby increasing attention and enhancing cognitive performance [19]. In long-term, regular MVPA fosters angiogenesis, neurogenesis, and synaptogenesis in brain areas that are important for the process of learning and memory [19–20]. Additionally, researchers verified both positive short and long-term changes in PA intervention programs for children [21–24]. These findings suggest that regular or additional PE lessons might stimulate children’s cognitive capacities (e.g. L2 learning of young refugees) within iPE-L2 approaches. Despite this evidence, no study has yet examined such effects among refugee children at primary school age.
We hypothesized that intPE-L2 learning activities in additional PE lessons may influence domain specific vocabulary learning, listening comprehension and use of local prepositions within the context of primary PE classes. In a quasi-experimental study, we therefore investigated the effectiveness of a school-based intPE-L2 learning approach for young refugees. We predicted improved domain specific vocabulary learning, listening comprehension and use of local prepositions after the intervention. The intervention was supposed to result in a better performance than mere schooling and maturation (control group). This study is the first of this kind in Germany and should therefore be regarded as an exploratory analysis.
Methods
Participants
Sixty-one young refugees took part in the study (34 males and 27 females, mean age 8.45 ± 1.37 years). The refugees were chosen from different primary schools in the city of Essen (North Rhine-Westphalia, Germany). Inclusion criterion for the study participation was immigration to Germany in consequence of fleeing conflict or persecution [25]. For reasons of practicability the participants could not be randomly assigned to the study conditions (intervention vs. control group). The intervention group (18 males and 13 females, aged 6–10 years), consisted of young refugees from six different countries (Spain, Iraq, Italia, Romania, Syria and Uganda). Students from this group immigrated to Germany between 2015 and 2016. 16 male and 14 female refugees (aged 6–11 years) from eleven different countries (Afghanistan, Azerbaijan, Spain, Greece, Italia, Nigeria, Syria, Romania, Bulgaria, Bosnia-Herzegovina, Croatia) served as controls. Students from this group immigrated to Germany between 2013 and 2016. The study was conducted in accordance to the declaration of Helsinki [26] and approved by the ethics committee of the Medical Faculty of the University of Duisburg-Essen (No. 17-7738-BO). All parents and students provided their written informed consent before taking part in the study.
Intervention
Procedure.
Trained teacher apprentices supervised the interventions. All of them obtained a degree in a pedagogic discipline (Bachelor of Education). Before the intervention started they received extensive training from the author and other staff members regarding the theoretical concepts and intervention contents. The intervention took place from August to December during the first school term in 2016/2017 and was announced as an additional physical education program for young refugees. Participants did not know about the study background. The intervention consisted of six lessons (duration: 90 minutes each). Language-enriched PE activities were included in all parts of the lessons, i.e., during warm up, main part, and during review and closure. Table 1 displays the specific contents during the intervention lessons. To ensure accuracy and preciseness of the intervention, a primary teacher was present during the intervention sessions and apprentices were asked to prepare a self-report after the lessons. Each individual intervention session was conducted by at least two of the apprentices. The control group did not receive any intervention and both, the intervention and the control group, were tested pre and post of the intervention period.
Intervention approach.
The intervention was based on an integrated Physical Education and L2-learning approach that combines PE integrated L2 learning strategies from the field of sport pedagogy [5,6] as well as major principles of usage-based approaches to second language acquisition [8].
Second language acquisition (SLA) is based on extensive L2 input. The more learners are exposed to the target language the more and the faster they will learn to use it [27]. Research from the field of SLA shows that L2 learners agree on the importance of frequency and richness of input for developing the highly implicit knowledge that is needed to communicate effectively in the L2 [8,28]. This way, L2 learners implicitly internalize the rules of the target language. Often used linguistic elements (e.g. vocabulary or abstract frames on sentence level) are learned faster than rarely used elements. These linguistic elements are pattern ranging from simple morphems like -ing to complex and abstract frames on sentence level like the imperative. Linguistic elements form a structured inventory of a speaker´s knowledge in the convention of their L2 (a semantic network).
As indicated by L2 research findings, SLA is an explicit and implicit learning process [29]. Explicit learning describes scenarios in which L2 learners are instructed to actively look for specific patterns (i.e., intentional learning). Implicit learning means that L2 students derive knowledge from complex, rule-governed stimuli without intending to or being aware of it. The intervention used in the present study was conceptualized as being an implicit and explicit learning scenario. Nevertheless, L2 learning can happen in diverse learning contexts in and outside school. Therefore, we selected linguistic elements that were specific enough to the PE content domain to allow the examination of intervention effects. Thus, we decided to focus on constructs on word level (i.e., domain specific vocabulary and use of local prepositions) and sentence level (i.e., listening comprehension) that are often used within the intervention context and that would have a low probability to be found outside the intervention context. The additional PE lessons were used as authentic and relevant opportunities to communicate. Furthermore, these communication opportunities were embedded and enriched with physical activities for stimulating L2 learning [12–17]. Against this background, we designed intervention lessons to support the use of local prepositions, providing diverse occasions for communicating, support movements and verbal activity (i.e., combining speaking and moving). The main goal of the intervention was to foster second language acquisition with the help of physical activity within additional L2-learning enriched PE lessons.
Outcome measures
To test students’ domain specific vocabulary learning, listening comprehension and use of prepositions we developed three test batteries using a multiple steps approach for language test development and examining strategies [30]. The whole process is represented in Fig 1.
Firstly, a list of typical substantives (→ domain specific vocabulary; e.g. “Weichbodenmatte” [engl.: soft flor mat]) and a list of typical imperatives (→ listening comprehension; e.g. “Dreh dich im Kreis” [engl.: spin in a cycle]) were developed. Also, we took many photos of a young girl standing in different positions next to a piece of gymnastics apparatus (e.g. a bench; → use of prepositions). To meet the strategy of rich input, the lists and figures contained linguistic elements (e.g. substantives), that would frequently occur within the context of primary PE games and would often be used during the intervention. Secondly, the lists and figures were checked by five PE experts (doctoral students, postdocs and professors) regarding their domain specificity and feasibility. At the end of this step, all linguistic elements were transferred into three standardized test batteries along with suitable verbal instructions. The test batteries for domain specific vocabulary and for use of prepositions were carried out by verbal and figural instructions. Thirdly, the instructions of all test batteries were checked for comprehensibility of language and clarity of figures. At the end of this step, ten domain specific vocabulary items, five listening comprehension items, and five prepositions items remained. In the next step, the final instrument was content validated by an expert from the field of second language acquisition research at the University of Duisburg-Essen and pilot tested. For this purpose, the instrument was utilized in three young refugees who had to show good usage of the instrument within the testing situation. Lastly, the author of the present study constructed a coding manual with rules for correct (= 1 point) and incorrect (= 0 point) answers for all outcome dimensions. The manual was developed based on the principles of standardized test items batteries and with the help of the audio recorded pilot tests to identify good examples and precedents for correct and incorrect answers. With the help of this manual 20% of the tests were double coded by two teacher trainees. Inter rater reliability for the coding was Cohen´s κ = .89 showing an almost perfect agreement [31]. The means of the outcome dimensions in pre-and post-tests were used to assess second language acquisition within the field of primary PE games.
Data analysis and control variables
All data were screened for missing data and outliers: across both sub-samples (intervention and control group), none of the variables had missing values.
The data analysis consisted of two parts: a preliminary analysis and a main analysis to discover the intervention effects. Descriptive statistical analysis was used to analyze the characteristics of the sample. A chi-square test for gender and a multivariate analysis of variance (MANOVA) for the other control variables were conducted with the collected pre-test data. This was done to examine statistical significant differences in the outcomes and control variables between the two groups prior to the intervention. Gender, age, basic motor competencies [32] and cognitive ability (the German version of SON-R 6–40; [33]) were chosen as control variables because they represent potentially important moderators in learning contexts (e.g. [34–37]). Although there are several cognitive ability tests that could be administered to a large group of people, we decided to use the SON-R 6–40 with an individual testing procedure because it is a fair assessment tool for children from cultural minorities [38]. It can be conducted without using written or spoken words, and it consists of four subtests (analogies, mosaics, categories and patterns). It is a valid and reliable intelligence test for children and adults from the age six to forty [33]. Further data (age, gender, immigration year to Germany) were recorded using a self-prepared questionnaire filled out by the children with the help of the teacher trainees, translators and the author of the study during audio-recorded face-to-face interview. Proficiency in the German language was judged by the class teachers at A1-Level (beginners) of the Common European Framework of Reference for Languages for all children within the intervention and control group. After immigrating to Germany, they all attended so called integration classes with intensive German language teaching (15-20h per week) in addition to their normal school routine.
The main analysis was performed to observe the intervention effects using linear mixed effect-models (LMM). We decided to use this class of statistical models to deal with dependency of a subject´s residuals that challenges researchers when using data from repeated measures and longitudinal studies. LMM use likelihood-based techniques to estimate means, variances, regression coefficients, and standard errors including random subject effects to account for the influence of subjects on their repeated observations. LMM can deal with balanced and unbalanced designs as well as autocorrelation by allowing to specify different base-lines and slopes for each of the children with regard of the dependent variables (further [39–40]).
Results
Preliminary analysis
The MANOVA results reflected no statistically significant differences at multivariate level (F(3, 57) = 1.34, p = .26; η2 = .07) for any of the investigated dependent variables. For the independent variables (see Table 2) significant differences were observed (F(1, 59) = 2.59, p = .04; η2 = .18). Subsequent ANOVAs yielded statistically significant differences regarding age (F(1, 59) = 8.96, p = .00; η2 = .13): the intervention group was younger (M = 7.96; SD = 1.16) than the control group (M = 8.96; SD = 1.40). Therefore, age was considered in the following analyses. No statistical significant differences were identified for cognitive ability (F(1, 59) = 0.68, p = .41; η2 = .01). Both groups presented similar ability values (intervention: M = 84.43; SD = 12.44; control: M = 81.20; SD = 15.11). Both basic motor competencies locomotion (F(1, 59) = 1.46, p = .23; η2 = .02) and object control (F(1, 59) = .24, p = .61; η2 = .00) did not show significant differences. Males and females were also equally represented in the intervention and control group (χ2(1) = .01; p< .05).
Effects of the intPE-L2 intervention
Descriptive analyses revealed differing baseline values between the conditions at pretests and differing error variances between and within conditions. Descriptive statistics of the outcome measures are displayed in Table 3. We performed a linear mixed effect model using R, Version 3.4.0 [41], and the function lmerTest from the packages lmerTest [42] to analyze the effectiveness of the intPE-L2 intervention. As fixed effects time (t1 = pretest, t2 = posttest], age and group (1 = IG, 2 = CG) as main effects and two interaction effects (group x time and age x group) were entered into the LMM. As random effects, intercepts and slopes as subjects for the effect of measurement were used. Tables 4–6 displays results of those dependent variables that are central for testing our hypothesis. The significance level was set at p≤ .05.
Domain specific vocabulary.
The linear mixed effect model (Table 4) revealed a main effect of time but not of age and group. Domain specific vocabulary scores in posttests increased significantly in both groups but this increase was not affected by age nor group. We found the predicted two-way interaction between time and group. The increase in test scores on domain specific vocabulary was pronounced in the intervention group compared to the control group. The interaction between age and group was not significant. The significant interaction of time of measurement and group is shown in Fig 2.
Listening comprehension.
The linear mixed effect model (Table 5) revealed no main effects of time, age and group. Listening comprehension scores after the intervention were not affected by group and by age. Crucially, we found the predicted two-way interaction between time and group. The increase in listening comprehension scores of the intervention group was more pronounced compared to the increase in the control group. The interaction between age and group was not significant. The significant interaction of time and group is shown in Fig 3.
Use of local prepositions.
The linear mixed effect model (Table 6) revealed no main effect of time, age and group. The use of local prepositions following the intervention period was not affected by group or by age. We did not find the predicted two-way interaction between time of measurement and group. Also, the interaction between age and group was not significant.
Discussion and conclusions
The aim of the present pilot study was to analyze second language acquisition effects of an intPE-L2 intervention for young refugees using a quasi-experimental design that referred to an established theoretical framework and included a control group. Our hypothesis postulated that the intervention would be associated with higher improvements regarding domain specific vocabulary learning, listening comprehension, and use of local prepositions within the context of primary PE games focusing on tossing and running rather than schooling and maturation only. The hypothesis was partly confirmed in accuracy measures in the domain specific vocabulary learning and listening comprehension tests where the intervention group showed better performance compared to the control group. No significant effect was found for the use of local prepositions. This suggests that the intPE-L2-intervention improved domain specific vocabulary and listening comprehension within the context of primary PE games for the target group beyond maturation and schooling. These findings are in line with earlier findings obtained from other L2-learners from several pre-school samples [9–11]. The missing effect on use of local prepositions beyond maturation and schooling may be due to the circumstance that teaching the use of local prepositions was not intense or domain specific enough to consistently affect this specific language skill. Furthermore, training the use of local prepositions is normally integrated in additional German language learning activities for this target group in extracurricular L2-learning lessons at this time of their school career. Therefore, we plan to explore in more detail to what extent the intPE-L2-intervention can improve the use of local and other prepositions beyond schooling and maturation in further studies. For this, the integration of more difficult test items might be beneficial to differentiate between groups. Also, video coding of intervention lessons could be used to examine the right amount of teaching intensity by analyzing the frequency of relevant elements (e.g. words, sentences). In addition, in future intPE-L2-interventions that will exceed the character of a pilot study we aim to test the children with a larger number of test items in domain specific vocabulary, listening comprehension and use of prepositions.
Albeit preliminary, the results of the present study suggest specific effects of intPE-L2-intervention on young refugees at primary school level on domain specific vocabulary learning and listening comprehension. Nevertheless, the findings should be treated with care, because they are grounded on a relatively small sample. Furthermore, the evidence is limited by the fact that the participants were not randomly assigned to the study groups. However, the pattern of results encourages further studies with larger randomized and controlled trials to test the replicability of effects and to test long-term effects. Also, it might be interesting to look at transfer effects in the future, i.e. whether the L2 learning that takes place during PE classes also affects the children’s performance during other school lessons. Beside successful replication, the mechanisms underlying these effects deserve further empirical investigation.
A validation of our results with refugees from different schools and other age groups might also be desirable. Future intPE-L2-intervention studies based on the same and other PE content might strengthen the evidence. It should also be noted that the present study was conducted with young refugees from diverse national, cultural, and religious backgrounds. If the results are successfully replicated in future studies and in view of the existing evidence concerning positive effects of intPE-L2-intervention, there is good reason to establish this approach as one effective domain specific L2-learning strategy within the PE context with the target group of refugees. Further studies should check whether these effects could also be transferred to intPE-L1-learning scenarios when participants do not have a L2-learning background.
References
- 1.
United Nations Higher Commissioner for Refugees, editors. Global trends. Forced displacement in 2016 [Internet]. Geneva, Switzerland: Author; 2017 [cited 2017 Aug 27]. Available from: http://www.unhcr.org/5943e8a34.
- 2.
United Nations Higher Commissioner for Refugees, editors. Statistical Yearbook 2014. 14th ed. Geneva, Switzerland: Author; 2015.
- 3.
Massumi M, von Dewitz N, Grießbach J, Terhart H, Wagner K, Hippmann K et al, editors. Bestandsaufnahme und Empfehlungen. Neu zugewanderte Kinder und Jugendliche im deutschen Schulsystem. Köln: Mercator; 2015.
- 4.
Ständige Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland. Interkulturelle Bildung und Erziehung in der Schule. Beschluss der Kultusministerkonferenz vom 25.10.1996 i.d.F. vom 05.12.2013 [Internet]. Bonn, Germany: Author; 1996 [cited 2017 Aug 23]. Available from: http://www.kmk.org/fileadmin/Dateien/veroeffentlichungen_beschluesse/1996/1996_10_25-Interkulturelle-Bildung.pdf.
- 5. Krüger M, Gebken U. Sportunterricht mit Seiteneinsteigern. Sportunterricht 2017;66(6):175–81.
- 6. Gebken U, Süßenbach J, Krüger M, van de Sand S. Sportunterricht mit neu zugewanderten Schülerinnen und Schülern. Sportpädagogik 2016;(3/4):58–61.
- 7. Cummins J. Cognitive/academic language proficiency, linguistic interdependence, the optimum age question and some other matters. Working Papers on Bilingualism 1979;19:121–9.
- 8.
Ellis N, Wulff S. Usage-based approaches to SLA. In: Van Patten B, Williams J, editors. Theories in Second Language Acquisition. An Introduction. New York: Routledge; 2015. p. 75–93.
- 9.
Zimmer R. Frühkindliche Bildung und Sport. In: Schmidt W. editor. Dritter deutscher Kinder- und Jugendsportbericht. Schorndorf: Hofmann; 2015. p. 395–415.
- 10. Derri V, Kourtessis T, Goti-Douma E, Kyrgiridis P. Physical Education and Language Integration: Effects on Oral and Written Speech of Pre-School Children. The Physical Educator 2010;67(4), 178–86.
- 11. Connor-Kuntz FJ, Dummer GM. Teaching across the curriculum: language-enriched physical education for preschool children. Adapt Phys Activ Q 1996;(133), 302–15.
- 12. Sibley BA, Etnier JL. The Relationship Between Physical Activity and Cognition in Children: A Meta-Analysis. Pediatr Exerc Sci 2003;15, 243–56.
- 13. Tomporowski PD, Davis CL, Miller PH, Naglieri JA. Exercise and Children´s Intelligence, Cognition, and Academic Achievement. Educ Psychol Rev 2008;20(2), 111–31. pmid:19777141
- 14. Fedewa AL, Ahn S. The effects of physical activity and physical fitness on children´s achievement and cognitive outcomes: a meta-analysis. Res Q Exerc Sport 2011;82(3), 521–35. pmid:21957711
- 15. Howie EK, Pate RR. Physical activity and academic achievement in children: A historical perspective. J Sport Health Sci 2012;1(3), 160–9.
- 16. Taras H. Physical activity and Student Performance at School. J Sch Health 2005;75(6), 214–8. pmid:16014127
- 17. Castelli DM, Hillmann CH, Buck SM, Erwin HE. Physical fitness and academic achievement in third- and fifth-grade students. J Sport Exerc Psychol 2007;29(2), 239–52. pmid:17568069
- 18. Jensen EP. A fresh look at brain-based education. Phi Delta Kappan 2008;89(6), 408–17.
- 19. Best JR. Effects of physical activity on children’s executive function: contributions of experimental research on aerobic exercise. Dev Rev 2010;30(4):331–51. pmid:21818169
- 20. Hillman CH, Castelli DM, Buck SM. Aerobic fitness and neurocognitive function in healthy preadolescent children. Med Sci Sports Exerc 2008;37(11):1967–74.
- 21. Davis CL, Tomporowski PD, McDowell JE. Exercise improves executive function and achievement and alters brain activation in overweight children: a randomized, controlled trial. Health Psychol 2008;30(1):91–8.
- 22. Hillman CH, Castelli DM, Buck SM. Aerobic fitness and neurocognitive function in healthy preadolescent children. Med Sci Sports Exerc. 2005;75, 175–85.
- 23. Mullender-Wijnsma MJ, Hartman E, de Greeff JW, Bosker RJ, Doolaard S., Visscher C. Improving academic performance of school-age children by physical activity in the classroom: 1-year program evaluation. J Sch Health 2015;85(6), 365–71. pmid:25877433
- 24. Mahar MT, Murphy SK, Rowe DA, Golden J, Shields AT, Raedeke TD. Effects of a classroom-based program on physical activity and on-task behavior. Med Sci Sports Exerc. 2006, 38(12), 2086–94. pmid:17146314
- 25.
United Nations High Commissioner for Refugees (1951). Convention and protocol relating to the status of refugees. Geneva: Switzerland.
- 26.
World Medical Association, editors. Declaration of Helsinki: ethical principles for research involving human subjects. [Internet]. Helsinki, Finland: Author; 2008 [cited 2017 Aug 22]. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/
- 27. Ellis R. Principles of instructed language learning. System 2005;33(2), 209–24.
- 28. Laufer B. Second Language vocabulary acquisition from language input and from form-focused activities. Language Teaching 2009;42(3), 341–54.
- 29.
Ellis R. Implicit and explicit knowledge about language. In: Cenoz J, Hornberger NH editors. Encyclopedia of language and education. Volume 6. Knowledge about language. Ney York: Springer. 2008. p. 1–13.
- 30.
The Council of Europe (2011). Manual for Language Test Development and Examining. Cambridge: UK.
- 31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1), 159–177. pmid:843571
- 32. Herrmann C, Gerlach E, Seelig H. Development and validation of a test instrument for the assessment of basic motor competencies in primary school. Meas Phys Educ Exerc Sci 2015;19(2), 80–90.
- 33.
Tellegen PJ, Laros JA, Petermann F. SON-R 6–40. Non-verbaler Intelligenztest. I. Technisches Manual. Göttingen: Hogrefe; 2012.
- 34.
Ellis L, Hershberger S, Field E, Wersinger S, Pellis S, Geary D, Palmer C, Hoyenga K, Hetsroni A, Karadi K, editors. Sex differences. Summarizing more than a century of scientific research. New York: Taylor and Francis; 2013.
- 35. Rindermann H. Increase of intelligence in childhood and youth. Psychologie in Erziehung und Unterricht 2011;58(3), 210–24.
- 36. Steinmayr R, Beauducel A, Spinath B. Do sex differences in a faceted model of fluid and crystallized intelligence depend on the method applied? Intelligence 2010;38(1), 101–10.
- 37. Arden R, Plomin R. Sex differences in variance of intelligence across childhood. Pers Individ Dif 2006;41(1), 39–48.
- 38. Renner G. Tellegen PJ, Laros JA, Petermann F (2012). SON-R 6–40 Non-verbaler Intelligenztest. Göttingen: Hogrefe. Testkoffer mit Auswertungsprogramm [Testbesprechung]. Psychologie in Erziehung und Unterricht 2013;60(4), 307–10.
- 39. Baayen RH, Davison DJ, Bates DM. Mixed-effects modeling with crossed random effects for subjects and items. J Mem Lang 2008;59(4), 390–412.
- 40. Gries T. The most under-used statistical method in corpus linguistics: multi-level (and mixed-effects) models. Corpora 2015;10(1), 95–125.
- 41.
R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria; 2017.
- 42. Kuznetsova A, Brockhoff PB, Christensen RHB. lmerTest: Tests in Linear Mixed Effects Models. R package version 2.0–33 [software]. 2016 [cited 2017 Aug 14]. Available from. https://CRAN.R-project.org/package=lmerTest.