| Abstract: |
Student ratings have become a standard instrument to efficiently assess students’ perceptions of their learning environment. Although student ratings are frequently used in research and educational practice, little is known about the relevance of the item addressee (“we/us” vs. “I/me”) for the information obtained. Even at first glance, measures of teaching quality substantially differ not only with regard to the assessed dimensions, but also with respect to the item addressee (e.g., Den Brok et al., 2006; Kuhlfeld, 2017; OECD, 2017). Theoretically, items worded with a we-addressee (“We feel motivated by the teacher”) or an I-addressee (“I feel motivated by the teacher”) encourage students to refer to different experiences. For instance, the item “We feel motivated by the teacher” refers to the perceptions of all students in the classroom, whereas the source of information for the item "I feel motivated by the teacher" is unique to each student. Only a few studies have investigated the role of the item addressee; however, these studies spoke in favor of its relevance (Den Brok et al.,2006; Fraser, 1995; McRobbie).
In the present study, we systematically examined whether the addressee in typical teaching quality measures makes a difference for the assessment of teaching quality at the student or classroom level. To do so, we experimentally varied the addressee of teaching quality items capturing six dimensions of teacher support in mathematics and German language arts classes. In total, we used data from 6,317 students in Grades 5 to 10 from 385 classes within 27 schools. We examined differences in the means of the two addressee versions, the level of agreement among students within classrooms, the associations between dimensions using the same addressee as well as different addressees, and correlations with a variety of student demographic variables such as socioeconomic status, academic self-concept, and grades.
Overall, we found only small differences between the two addressee versions in both subjects. The results showed that using the we-addressee resulted in more positive ratings of teaching quality. Furthermore, correlations with students’ demographic variables tended to be higher at the student level when using an I-addressee and higher at the classroom level when using a we-addressee. Only small differences were found between addressee versions in students’ relative agreement, intercorrelations of different dimensions using the same addressee, and intercorrelations of the same dimensions using different addressees. From a practical perspective, the findings suggest that the choice of addressees matters but is less influential than one might expect.
|