Skip to main content
Published Online:https://doi.org/10.1024/1010-0652/a000232

Zusammenfassung. Die vorliegende Untersuchung überprüft die Konstruktvalidität einer 16 Items umfassenden Kurzversion der Integrated Teacher Report Form (ITRF), einem universellen und unterrichtsrelevanten Verhaltensscreening zur Diagnostik des externalisierenden Verhaltens von Schüler_innen im Klassenraum. 107 Lehrkräfte bearbeiteten für insgesamt 1048 Schülerinnen und Schüler der ersten bis sechsten Klasse die ITRF sowie zusätzlich jeweils eines von drei im deutschsprachigen Raum etablierten Beurteilungsverfahren. Die Analyse der konvergenten und diskriminanten Validität erfolgt anhand einer Multitrait-Multimethod (MT-MM) Korrelationsmatrix sowie einem strukturprüfenden Correlated Trait-Correlated Method minus 1 [CT-C(M-1)] Modell zur separaten Analyse des Einflusses der Konstrukte (lernbezogene / aufmerksame Verhaltensprobleme, oppositionelle / störende Verhaltensprobleme) und der Methoden (ITRF, zusätzliches Beurteilungsverfahren) auf die erzielten Werte der Beurteilungen. Die Ergebnisse zeigen, dass sich die Stärke der theoretisch postulierten Korrelationen mit den empirischen Daten erwartungskonform abbilden lassen, was auf konvergente und diskriminante Validität hinweist. Die Varianz der ITRF-Werte lässt sich zu einem größeren Anteil durch das zu messende Konstrukt als durch methodenspezifische Einflüsse erklären. Somit liefern unsere Befunde Evidenz für eine angemessene Konstruktvalidität des Verfahrens, weshalb sich die Kurzversion der ITRF für den praktischen Schuleinsatz eignet.


Construct Validity of a Universal Screener to Economically Assess Students' Behavior in the Classroom – a Multitrait-Multimethod-Analysis

Abstract. This study examines the construct validity of a 16-item version of the Integrated Teacher Report Form (ITRF), a universal screening tool to assess students' externalizing behavior problems in the classroom. The ITRF as well as another of three established screening tools were completed for 1048 first- to sixth-graders by 107 teachers. The Multitrait-Multimethod (MT-MM) correlation matrix was used to analyze convergent and discriminant validity. In addition, the Correlated Trait-Correlated Method minus 1 [CT-C(M-1)] model was used to analyze the influence of the constructs (Academic Productivity Problems, Oppositional / Disruptive Behavior) and the methods (ITRF, additional screener) on the variance. The MT-MM matrix indicate both convergent and discriminant validity. The variance in the ITRF scores attributable to the underlying construct was higher than the variance attributable to method specific influences. Therefore, the screener is suitable for practical interpretation and use.

Literatur

  • Albers, C. A., Glover, T. A. & Kratochwill, T. R. (2007). Introduction to the special issue: How can universal screening enhance educational and mental health outcomes? Journal of School Psychology, 45, S. 113 – 116. https://doi.org/10.1016/j.jsp. 2006.12.002 First citation in articleCrossrefGoogle Scholar

  • Becker, A., Woerner, W., Hasselhorn, M., Banaschewski, T. & Rothenberger, A. (2004). Validation of the parent and teacher SDQ in a clinical sample. European Child and Adolescent Psychiatry, 13, S. 11 – 11. First citation in articleCrossrefGoogle Scholar

  • Beelmann, A. (2008). Prävention im Schulalter. In B. Gasteiger-KlicperaH. JuliusC. Klicpera (Hrsg.), Sonderpädagogik der sozialen und emotionalen Entwicklung (S. 442 – 464). Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Bettge, S., Ravens-Sieberer, U., Wietzker, A. & Hölling, H. (2002). Ein Methodenvergleich der Child Behavior Checklist und des Strengths and Difficulties Questionnaire. Das Gesundheitswesen, 64, S. 119 – 124. https://doi.org/10.1055/s-2002-39264 First citation in articleCrossrefGoogle Scholar

  • Borg, I. & Shye, S. (1995). Facet theory: Form and content. Thousand Oaks: Sage. First citation in articleGoogle Scholar

  • Briesch, A. M., Casale, G., Grosche, M., Volpe, R. J. & Hennemann, T. (2017). Initial Validation of the Usage Rating Profile-Assessment for Use within German Language Schools. Learning Disabilities – A Contemporary Journal, 15, S. 193 – 207. First citation in articleGoogle Scholar

  • Briesch, A. M., Swaminathan, H., Welsh, M. & Chafouleas, S. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52, S. 13 – 35. https://doi.org/10.1016/j.jsp. 2013.11.008 First citation in articleCrossrefGoogle Scholar

  • Bruhn, A. L., Woods-Groves, S. & Huddle, S. (2014). A Preliminary Investigation of Emotional and Behavioral Screening Practices in K-12 Schools. Education & Treatment of Children, 37, S. 611 – 634. First citation in articleCrossrefGoogle Scholar

  • Campbell, D. T. & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, S. 81 – 105. https://doi.org/10.1037/h0046016 First citation in articleCrossrefGoogle Scholar

  • Casale, G., Volpe, R. J., Daniels, B., Hennemann, T., Briesch, A. M. & Grosche, M. (2018). Measurement Invariance of a Universal Behavioral Screener across Samples from the USA and Germany. European Journal of Psychological Assessment, 34, S. 87 – 100. First citation in articleLinkGoogle Scholar

  • Cohen, J. (1992). Statistical power analysis. Current directions in psychological science, 1, S. 98 – 101. First citation in articleCrossrefGoogle Scholar

  • Costello, E. J., Egger, H. & Angold, A. (2005). 10-Year Research Update Review: The Epidemiology of Child and Adolescent Psychiatric Disorders: I. Methods and Public Health Burden. Journal of the American Academy of Child & Adolescent Psychiatry, 44, S. 972 – 986. https://doi.org/10.1097/01.chi.0000172552.41596.6f First citation in articleCrossrefGoogle Scholar

  • Daniels, B., Volpe, R. J., Briesch, A. M. & Fabiano, G. A. (2014). Development of a problem-focused behavioral screener linked to evidence-based intervention. School Psychology Quarterly, 29, S. 438 – 451. https://doi.org/10.1037/spq0000100 First citation in articleCrossrefGoogle Scholar

  • Daniels, B., Volpe, R. J., Fabiano, G. A. & Briesch, A. M. (2017). Classification Accuracy and Acceptability of the Integrated Screening and Intervention System Teacher Rating Form. School Psychology Quarterly, 32, S. 212 – 225. https://doi.org/10.1037/spq0000147 First citation in articleCrossrefGoogle Scholar

  • Döpfner, M., Kinnen, C., Weber, K. N. & Plück, J. (2011). Verhaltensauffälligkeiten von Grundschulkindern: Ergebnisse zur deutschen Fassung des Lehrerfragebogens über das Verhalten von Kindern und Jugendlichen (TRF). Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 43, S. 99 – 107. https://doi.org/10.1026/0049-8637/a000039 First citation in articleLinkGoogle Scholar

  • Döpfner, M., Plück, J. & Kinnen, C. (2015). CBCL / 6 – 18R, TRF / 6 – 18R, YSR / 11 – 18R Deutsche Schulalter-Formen der Child Behavior Checklist von Thomas M. Achenbach. Göttingen: Hogrefe. First citation in articleGoogle Scholar

  • Dowdy, E., Dever, B. V., Raines, T. C. & Moffa, K. (2016). A preliminary investigation into the added value of multiple gates and informants in universal screening for behavioral and emotional risk. Journal of Applied School Psychology, 32, S. 178 – 198. First citation in articleCrossrefGoogle Scholar

  • Eid, M., Lischetzke, T., Nussbeck, F. W. & Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8, S. 38 – 60. https://doi.org/10.1037/1082-989X.8.1.38 First citation in articleCrossrefGoogle Scholar

  • Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M. & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, S. 230 – 253. https://doi.org/10.1037/a0013219 First citation in articleCrossrefGoogle Scholar

  • Eklund, K., Renshaw, T. L., Dowdy, E., Jimerson, S. R., Hart, S. R., Jones, C. N. & Earhart, J. (2009). Early Identification of Behavioral and Emotional Problems in Youth: Universal Screening versus Teacher-Referral Identification. California School Psychologist, 14, S. 89 – 95. First citation in articleCrossrefGoogle Scholar

  • Fabiano, G. A., Vujnovic, R., Naylor, J., Pariseau, M. & Robins, M. (2009). An Investigation of the Technical Adequacy of a Daily Behavior Report Card (DBRC) for Monitoring Progress of Students with Attention-Deficit / Hyperactivity Disorder in Special Education Placements. Assessment for Effective Intervention, 34, S. 231 – 241. https://doi.org/10.1177/1534508409333344 First citation in articleCrossrefGoogle Scholar

  • Fletcher, J. M., Coulter, W. A., Reschly, D. J., & Vaughn, S. (2004). Alternative approaches to the definition and identification of learning disabilities: Some questions and answers. Annals of dyslexia, 54, S. 304 – 331. First citation in articleCrossrefGoogle Scholar

  • Forness, S. R., Kim, J., & Walker, H. M. (2012). Prevalence of Students with EBD: Impact on General Education. Beyond Behavior,21, S. 3 – 10. First citation in articleGoogle Scholar

  • Glover, T. A. & Albers, C. A. (2007). Considerations for evaluating universal screening assessments. Journal of School Psychology, 45, S. 117 – 135. First citation in articleCrossrefGoogle Scholar

  • Gomez, R. & Gomez, A. (2015). Agreement of adolescent ratings with mother ratings and teacher ratings of ADHD symptom groups: A correlated trait-correlated method minus one analysis. Personality And Individual Differences, 82, S. 131 – 135. doi:10.1016/j.paid.2015.02.015 First citation in articleCrossrefGoogle Scholar

  • Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, S. 581 – 586. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x First citation in articleCrossrefGoogle Scholar

  • Gräsel, C. & Parchmann, I. (2004). Implementationsforschung – oder: der steinige Weg, Unterricht zu verändern. Unterrichtswissenschaft, 32, S. 196 – 214. First citation in articleGoogle Scholar

  • Grosche, M., Volpe, R. J., Hennemann, T. & Casale, G. (2015). Interkulturelle Entwicklung und Evaluation eines sozial validen, universellen Verhaltensscreenings in Schulen [Forschungsnotiz]. Erziehungswissenschaft, 16, S. 129 – 131. First citation in articleGoogle Scholar

  • Hölling, H., Schlack, R., Petermann, F., Ravens-Sieberer, U., Mauz, E. & Group, K. S. (2014). Psychische Auffälligkeiten und psychosoziale Beeinträchtigungen bei Kindern und Jugendlichen im Alter von 3 bis 17 Jahren in Deutschland – Prävalenz und zeitliche Trends zu 2 Erhebungszeitpunkten (2003 – 2006 und 2009 – 2012). Bundesgesundheitsblatt – Gesundheitsforschung – Gesundheitsschutz, 57, S. 807 – 819. https://doi.org/10.1007/s00103-014-1979-3 First citation in articleCrossrefGoogle Scholar

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6, S. 1 – 55. First citation in articleCrossrefGoogle Scholar

  • Kauffman, J. M. (1999). How We Prevent the Prevention of Emotional and Behavioral Disorders. Exceptional Children, 65, S. 448 – 468. https://doi.org/10.1177/001440299906500402 First citation in articleCrossrefGoogle Scholar

  • Kultusministerkonferenz(2016). Sonderpädagogische Förderung in Schulen 2003 – 2014. Sekretariat der Ständigen Konferenz der Länder in der Bundesrepublik Deutschland. Verfügbar unter https://www.kmk.org fileadmin/Dateien/pdf/Statistik/Dokumentationen/Dok_210_SoPae_2014.pdf. Aufgerufen am 12. September 2017. First citation in articleGoogle Scholar

  • Landerman, L. R., Land, K. C. & Pieper, C. F. (1997). An Empirical Evaluation of the Predictive Mean Matching Method for Imputing Missing Values. Sociological Methods & Research, 26, S. 3 – 33. https://doi.org/10.1177/0049124197026001001 First citation in articleCrossrefGoogle Scholar

  • Lane, K. L., Robertson Kalberg, J., Lambert, E. W., Crnobori, M. & Bruhn, A. L. (2010). A Comparison of Systematic Screening Tools for Emotional and Behavioral Disorders: A Replication. Journal of Emotional and Behavioral Disorders, 18, S. 100 – 112. https://doi.org/10.1177/1063426609341069 First citation in articleCrossrefGoogle Scholar

  • Li, C. H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48, S. 936 – 949. First citation in articleCrossrefGoogle Scholar

  • Little, T. D., Cunningham, W. A., Shahar, G. & Widaman, K. F. (2002). To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits. Structural Equation Modeling, 9, S. 151 – 173. First citation in articleCrossrefGoogle Scholar

  • Little, T. D., Rhemtulla, M., Gibson, K. & Schoemann, A. M. (2013). Why the items versus parcels controversy needn't be one. Psychological Methods, 18, S. 285 – 300. https://doi.org/10.1037/a0033266 First citation in articleCrossrefGoogle Scholar

  • Lüdtke, O., Robitzsch, A., Trautwein, U. & Köller, O. (2007). Umgang mit fehlenden Werten in der psychologischen Forschung. Psychologische Rundschau, 58, S. 103 – 117. https://doi.org/10.1026/0033-3042.58.2.103 First citation in articleLinkGoogle Scholar

  • Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, S. 335 – 361. First citation in articleCrossrefGoogle Scholar

  • Mealli, F. & Rubin, D. B. (2015). Clarifying missing at random and related definitions, and implications when coupled with exchangeability: Table 1. Biometrika, 102, S. 995 – 1000. https://doi.org/10.1093/biomet/asv035 First citation in articleCrossrefGoogle Scholar

  • Muthén, B. & Muthén, L. (1998 – 2012). Mplus Version (Version 7). First citation in articleGoogle Scholar

  • Nußbeck, S. (2001). Zum sogenannten Paradigmenwechsel in der sonderpädagogischen Diagnostik. Sonderpädagogik 31, S. 46 – 52. First citation in articleGoogle Scholar

  • Petermann, U. & Petermann, F. (2013). Lehrereinschätzliste für Sozial- und Lernverhalten (2. Aufl.). Stuttgart: Hogrefe. First citation in articleGoogle Scholar

  • Reinke, W. M., Herman, K. C., Petras, H. & Ialongo, N. S. (2008). Empirically Derived Subtypes of Child Academic and Behavior Problems: Co-Occurrence and Distal Outcomes. Journal of Abnormal Child Psychology, 36, S. 759 – 770. https://doi.org/10.1007/s10802-007-9208-2 First citation in articleCrossrefGoogle Scholar

  • Riley-Tillman, T. C., Chafouleas, S. M., Christ, T., Briesch, A. M. & LeBel, T. J. (2009). The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs). School Psychology Quarterly, 24, S. 1 – 12. https://doi.org/10.1037/a0015248 First citation in articleCrossrefGoogle Scholar

  • Rogge, J., Koglin, U. & Petermann, F. (2018). Do They Rate in the Same Way? Testing of Measurement Invariance Across Parent and Teacher SDQ Ratings. European Journal of Psychological Assessment, 34, S. 69 – 78. First citation in articleLinkGoogle Scholar

  • Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. John Wiley & Sons, Inc., New York. First citation in articleCrossrefGoogle Scholar

  • Saile, H. (2007). Psychometrische Befunde zur Lehrerversion des „Strengths and Difficulties Questionnaire“ (SDQ-L). Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 39, S. 25 – 32. https://doi.org/10.1026/0049-8637.39.1.25 First citation in articleLinkGoogle Scholar

  • Salvia, J., Ysseldyke, J.E. & Bolt, S. (2010). Assessment in special and inclusive education, 11th Edition. Boston, MA: Houghton Mifflin. First citation in articleGoogle Scholar

  • Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, S. 347 – 363. First citation in articleCrossrefGoogle Scholar

  • Schmidt-Atzert, L. & Amelang, M. (2012). Psychologische Diagnostik (5. Aufl.). Berlin Heidelberg: Springer. First citation in articleCrossrefGoogle Scholar

  • Sparfeldt, J. R., Rost, D. H., Schleebusch, R. & Heise, A.-L. (2012). Lehrerbeurteiltes Schülerverhalten. Eine Evaluation der „Lehrereinschätzliste für Sozial- und Lernverhalten“ (LSL). Psychologie in Erziehung und Unterricht, 59, S. 146 – 158. First citation in articleGoogle Scholar

  • Tabachnick, B. G. & Fidell, L. S. (2007). Using Multivariate Statistics (3. Aufl.). New York: Pearson. First citation in articleGoogle Scholar

  • Volpe, R. J., Briesch, A. M. & Chafouleas, S. M. (2010). Linking Screening for Emotional and Behavioral Problems to Problem-Solving Efforts: An Adaptive Model of Behavioral Assessment. Assessment for Effective Intervention, 35, S. 240 – 244. https://doi.org/10.1177/1534508410377194 First citation in articleCrossrefGoogle Scholar

  • Volpe, R. J., Casale, G., Mohiyeddini, C., Grosche, M., Hennemann, T., Briesch, A. M. & Daniels, B. (2018). A universal behavioral screener linked to personalized classroom interventions: Psychometric characteristics in a large sample of German schoolchildren. Journal of School Psychology, 66, S. 25 – 40. First citation in articleCrossrefGoogle Scholar

  • Volpe, R. J. & Fabiano, G. A. (2013). Daily Behavior Report Cards: An Evidence-Based System of Assessment and Intervention. New York: Guilford Press. First citation in articleGoogle Scholar