Profiles of Good Teaching Practices in STEM Disciplines: An Analysis of Mixed Methods of Academic and Assessment Variables of Teaching in the First Cycle of Civil Engineering

Introduction

University fulfills a relevant role in society because it is a key factor in the formation of advanced human capital, production of new knowledge, innovation, and transference. In this respect, the mission of the university in Chilean society is demonstrated in the functions it fulfills research and teaching, both associated with the country’s socio-economic development (Mondaca et al., 2019). What underlies these functions is a quality education offered according to global demands and needs, moving toward the insertion of knowledge into society to respond to the demands of social, productive, scientific, and cultural development (SIES, 2014).

This leads us to review the concept of quality in education. Education quality emerged from the World Conference on Education for All (UNESCO, 1990, cited in OECD, 2018), in which the predominant quantitative direction (schooling) of the right to education changes to a qualitative direction, supported and disseminated by the business world (World Bank, 2007, cited in OECD, 2018) and, that, to date, is the frame of reference for education policies internationally (OECD, 2018, 2019). In this perspective, the concept of education quality is underpinned by the different educational institutions, functions, roles, teaching-learning processes, the teacher’s role, etc., which constitute an appraisable reality to define quality (OECD, 2018), as well as student learning, the performance of instructors and heads, the efficiency of the investment, community outreach, etc. In addition, it includes standards, improvement policies, and measurement of the progress carried out by the different education systems internationally (Prieto and Manso, 2018).

From this perspective, in the context of the Chilean political process of the last few decades, characterized by a deep social malaise, a series of demands and vindications has been made in the student context characterized by differing degrees of violence (Asún et al., 2019; Montoni Rios, 2019), the substantive criticism of which is placed on the neoliberal model (Durán, 2018), its consequences in the segmentation of academic outcomes in the school and university systems, in addition to its implications of income inequality and social segregation (Donoso, 2013; Murillo and Martínez-Garrido, 2017). Thus, the Chilean education model has formed part of the political dispute until today, and its demands have triggered deep changes.

In this light, we ask ourselves if student evaluations can point to the important topics of teaching that permit feedback on the teaching and learning processes in the STEM areas: science, technology, engineering, and mathematics. We even wonder if the instruments we have today to evaluate teaching in Chilean state universities can respond to the dimension of teachers and their skills and difficulties.

If we are not able to observe ourselves critically and in a complex way in this challenge to offer better education quality in STEM, we will be unable to evaluate and redefine ourselves effectively. We will continue, therefore, justifying the complaints that have already been raised in our unequal society, and we will be incapable of offering a way to move toward a quality STEM education.

Science, Technology, Engineering, and Mathematics Programs in the Context of Education Quality

At the moment, a relevant area to improve the quality of undergraduate education are the STEM disciplines (Dennin et al., 2017). These disciplines have shown a drop in student interest and participation internationally, affecting women in particular (Kemp et al., 2021).

In the United States, these disciplines are presenting a possible shortage of professionals together with the future workforce, which makes it necessary to attract and retain STEM students in higher education (Robert and Carlsen, 2017). It is also a general need in Europe, where the European Union is making efforts to prioritize STEM fields, to increase the number of degree holders in these areas, and to prevent strategic sectors of the economy from being left without qualified professionals (Peña-Calvo et al., 2016). In Latin America, the situation is similar (Vázquez-Alonso and Manassero-Mas, 2016).

From this perspective, education-related subjects and poor education quality correspond to the high failure and dropout rates in these disciplines (Paterson, 2017; Dewberry and Jackson, 2018). Universities have implemented models to prevent undergraduates from dropping out generally (McGinn and Schiefelbein, 2015; Schiefelbein, 2017). The studies on student dropouts in STEM disciplines, with analyses on perceived supports and barriers, report that companions and family are the most important perceived supports, whereas the greatest perceived barriers include economic difficulties and teachers (Peña-Calvo et al., 2016).

The trend in the development of the academic program worldwide reveals a strong incentive for academics to focus on research. University faculty members are usually assessed and promoted mainly based on the success of their research (Bradforth et al., 2015; Dennin et al., 2017). This phenomenon has caused a strong relegation of teaching, “I think it is fair to say that in the academic profession, teaching is one of the least professional dimensions of the university” (Weiler, 2017). In the case of Chile, state economic incentives are distributed among universities in terms of their scientific production. This has meant that academics focus a good part of their energy on this area, taking away interest from teaching. The incentives, as analyzed by Quezada-Hofflinger and Vallejos-Romero (2018), are put into publishing for the Chilean case, but it is no different for the American and European contexts. The result is the same, “a series of incentives that overlook teaching” (Weiler, 2017, p. 879), resulting in a faculty that teaches as best they can. In this respect, Weiler (2017), in a mature reflection of his academic career, questioned ironically the lack of seriousness that universities give to teacher training: “But consider the following, almost universal, paradox. To receive a position as a kindergarten teacher, an elementary school teacher, or a high school teacher, in most jurisdictions the applicant would have to have undergone specialized training – in addition to any subject-matter university degree, he or she may have earned– to occupy a position of such individual and collective responsibility. The exception? University teachers.” (Weiler, 2017, p. 877). University teaching, as Weiler (2017) exemplified, is a kind of erratic exercise and in the best case supported by high intuition, experience, and natural vocation.

Teaching Science, Technology, Engineering, and Mathematics Programs

In their research on improving, undergraduate mathematics teaching focused on teachers, Schoenfeld et al. (2016) surmise that even among those teachers identified as good, some lack formal teaching preparation. According to Robert and Carlsen (2017), a literature review on STEM teachers reports the difficulties that teachers express in integrating the responsibilities of teaching and research.

In this context, to improve the quality of undergraduate teaching in general, not only in the STEM disciplines, it is necessary to assess teaching and to have consistency between established teaching policies and practices (Dennin et al., 2017). In this sense, the training that teachers receive fulfills an important role, because studies report that institutions cover training focused on research, but very little or no preparation for teaching (Robert and Carlsen, 2017; Connolly et al., 2018).

From this perspective, the literature has demonstrated generic aspects of teaching in higher education and also aspects of teaching in STEM disciplines, such as the learning process and student-centered teaching practices (Brown et al., 2014, cited in Dennin et al., 2018; Kober, 2015; Trenshaw et al., 2016), widely showing the significance of active learning pedagogies in students’ learning in general and, in particular for students in STEM programs, over the traditional methods with lectures (Trenshaw et al., 2016). Likewise, one important factor is the pedagogical knowledge that instructors possess, i.e., the knowledge about teaching and learning (e.g., such as knowledge of learning theory, classroom management, and student motivation) (Auerbach and Andrews, 2018). In this sense, the importance of pedagogical content knowledge (Gess-Newsome, 2015) is highlighted, especially in an international context that uses standards to measure knowledge and skills, in addition to considering inclusive pedagogies that consist of specific guidelines for STEM disciplines, such as modeling a variety of pedagogies and collaborating in the development of student-centered pedagogies (Borda et al., 2020).

In the line of student-centered learning (SCL), a proven effective teaching strategy, Sabah and Du (2018) showed that the real practices of instructors prioritize content delivery, control over the class by the teachers, the definition of student learning activities, and summative assessment. Also, it was demonstrated that among the limited practices were student-student interactions and formative assessment.

It is important to highlight that the recent international literature understands feedback as the student taking an active role in seeking, generating, and using feedback (communicative and dialogic model) (Van der Kleij et al., 2019), rather than the transmission of information from the teacher to the student. This is also perceived positively by university teachers and has a great effect on their teaching, collaborating with the improvement of courses (Flodén, 2017). In this regard, surveys promote an obsolete view of feedback, conceiving it as information transmitted from teacher to student in a timely and specific manner, leaving aside the student’s role in this process (Winstone et al., 2021).

Other studies focus on: feedback in teaching or peer coaching (instead of peer observation and review) (Gormally et al., 2014), implementing peer observation of faculty to promote inquiry-based learning of the practices (Dillon et al., 2020), incorporating activities integrated into the teaching practice in the STEAM framework (A as in Arts), especially the integration of maker education (Juškevičienė et al., 2020).

Pedagogies associated with student learning have sufficient evidence; however, there is a paucity of knowledge about how university professors acquire pedagogical competence in STEM contexts. A critical literature review conducted by Winberg et al. (2019) on the topic revealed the absence of focus on STEM disciplines in themselves; the authors added that the key subject of what makes STEM disciplines difficult to learn and challenging to teach is not addressed either. This is to say, there is a lack of studies that identify “the type of professional learning that would allow STEM university professors to provide ‘epistemological access’ (Morrow, 2009, cited in Winberg et al., 2019) to STEM knowledge: to its logic, systems, processes and values” (p. 11). In this sense, the authors emphasize that too much emphasis has been placed on generic teaching methods (which represents only a part of STEM pedagogical competence) and the specific teaching practice of the discipline has been neglected. This, the authors suggest, maybe due to “it being presumed that STEM university professors have experience in STEM, but this supposition must be questioned” (p. 11).

From the management area, a paper by the Association of American Universities (AAU) in the United States in the context of improving the effectiveness of undergraduate STEM education showed the essential and strategic role of the academic department and department head in improving the quality of undergraduate education. This indicates how the actions carried out by department heads affect the everyday experience of teachers, staff, and students, being responsible for 80 percent of the administrative decisions on campus. In this sense, the important actions to take include investing in faculty members who have a profile of experience in STEM issues, a deep understanding of effective pedagogy, and experience in the use of evidence-based teaching practices. From the perspective of effective use of this personnel, these members need to have a voice in departmental decision-making about teaching processes and curricula (Coleman et al., 2019).

Assessing the Effectiveness of Science, Technology, Engineering, and Mathematics Education

In this sense, university teaching comprises all the activities undertaken to train the professionals and researchers society demands, and as an assessable reality, it can be measured through different indicators (Mondaca et al., 2019). One way is to evaluate teaching effectiveness through surveys applied to students at the end of a course. However, the literature reports that such evaluations, student evaluations of teaching (SET), hardly serve this purpose. Braga et al. (2014) report that in such evaluations the teacher’s effectiveness is correlated negatively with the students’ evaluations, supporting the idea that the students assess the professors based on the gain realized.

Therefore, assuming the need to improve teaching practices in the STEM context, this study is meant to investigate STEM teaching practices and their relationship with student academic variables.

Materials and Methods

Study Aim, Design, and Context

The study aimed to determine profiles of good teaching practices based on teaching evaluation variables and student academic context variables in STEM disciplines.

The design was mixed methods based on secondary data. Mixed methods make it possible to explore quantitative and qualitative data synergically, contributing a greater understanding of the problem being studied (Creswell and Plano-Clark, 2011). The study was conducted in a state university in southern Chile, framed in the first cycle of STEM disciplines, in the Faculty of Engineering and Sciences, using data from 2016 to 2017.

Description of the Materials

Secondary Database

The database with information on the subjects with their respective performance indicators and the results of the teaching evaluations from 2016 to 2017 of all the academics in the basic sciences departments in the Faculty of Engineering and Sciences that offered basic cycle courses in the Civil Engineering programs was requested from the Undergraduate Academic Office. That office assigned a code to the academics to guarantee anonymity. This study was approved by the Science Ethics Committee of the university.

Thus, the database includes: (a) Data from 2016, with 147 teachers evaluated in a total of 35 courses, 31 of them modular with a minimum of 2 modules and a maximum of 14 modules; (b) Data from 2017, with 163 teachers evaluated in a total of 37 courses, 33 of modular with a minimum of 2 modules and a maximum of 13 modules. The average number of students registered per course was 40 (SD = 21.4) for 2016 and 37 (SD = 21.6) for 2017, with a minimum of two and a maximum of 89 students; (c) Academic context data of students by course; and (d) Data of comments by the students, 947 comments (2016 and 2017).

Instrument

The Teaching Evaluation survey consists of 23 items measured on a Likert-type scale of 1 to 5 points (1 = strongly disagree, to 5 = strongly agree) and seven dimensions with weights of 10, 10, 20, 20, 10, 10, and 20%, respectively. The overall academic evaluation score is obtained by the weighted sum of each dimension, i.e., organization and responsibility (4 items), instruction and clarity (2 items), student motivation and participation (3 items), learning achieved (4 items), interpersonal relation (2 items), assessments (4 items), and comprehensive education (4 items). The general scale presented high internal reliability (Cronbach’s alpha of 0.91). Additionally, the scale presented adequate reliability values for the 7 dimensions of the scale (Cronbach’s alpha of 0.69, 0.67, 0.72, 0.82, 0.86, 0.79, 0.78, respectively) (DICDO, 2015). This instrument ends with a section for student comments in free format and is non-obligatory. The goal is to get feedback from the students on their perception of the instructors’ teaching practices.

This instrument was applied to the student body electronically via the institutional Intranet; their responses were anonymous and obligatory. It was administered before the end of the course, partly to avoid the bias associated with the final grade in the course.

Processes and Methodologies Used

The database was cleaned, identifying lost values and response patterns. In the comments section, 919 student comments (2016 and 2017) were coded using NVivo software. The codes were grouped according to three criteria: year, semester, and discipline (mathematics, chemistry, and physics).

Type of Analysis Used

Quantitative Analysis

A cluster analysis was performed to gain a descriptive-comprehensive view of the types of teachers who taught basic sciences courses in 2016 and 2017, according to the variables: number of students enrolled in the courses, number of students taking the course for the first time, pass percentage, average grade obtained in the course, number of women enrolled in the course, number of men enrolled in the course, average grade in organization and responsibility, instruction and clarity, student motivation and participation, learning achieved, interpersonal relation, assessments, and comprehensive education. As a grouping algorithm, Ward’s hierarchical clustering was used. The similarity measure between objects was squared Euclidean distance (Hair et al., 2004). This analysis was performed with the JMP14 cluster procedure. The number of groups was obtained by observing the dendrogram, which was confirmed by the percentage change in the recomposed cluster coefficients. To distinguish differences between the groups of continuous variables, an analysis of variance was carried out (99% confidence level). Since Levene’s statistic indicated non-homogeneous variances in all the continuous variables analyzed, the variables for which the analysis of variance resulted in significant differences (P ≤ 0.001) were subjected to Dunnett’s T3 multiple comparisons test. Finally, the clusters were characterized by including the dimensions that assess the teaching evaluation instrument concerning teaching practices.

Qualitative Analysis

To analyze the information contained in the comments, we used qualitative-interpretative content analysis. The first cycle was an initial coding that contained in vivo and structural codes. The second coding cycle was oriented to eclectic coding (Charmaz, 2014; Flick, 2014; Saldaña, 2016). The comments were organized into twelve analysis units under the following criteria: we grouped the mathematics courses and divided them into four analysis units. Two mathematics courses for 2016 (year) and a “1” for those in the first semester or a “2” for those in the second semester. Thus, we had, for example, Mathematics 2016-1. We repeated the same process for the courses in Physical Sciences and Chemistry.

From the eclectic coding, a series of codes were constructed that condensed what was analyzed during the first coding, and these were used as the basis to once again code each analysis unit group organized as indicated previously by year, semester, and discipline: F1-2016; F2-2016; F1-2017; F2-2017 (Physics, semester, year); M1-2016; M2-2016; M1-2017; M2-2017 (Mathematics, semester, year), and Q1-2016; Q2-2016; Q1-2017; Q2-2017 (Chemistry, semester, and year).

Results

As has been described, for this study quantitative and qualitative data were obtained from a secondary database comprised of teaching evaluation variables in the basic science subjects in the Faculty of Engineering and Sciences from 2016 to 2017, data from student variables, and data from student comments. The study aimed to determine profiles of good teaching practices based on teaching evaluation variables and student academic context variables in STEM disciplines. The interest in using mixed methods was to broaden and present complementary findings and deeper knowledge of the area being studied, i.e., teaching practices in the STEM context.

Results of the Quantitative Data

The cluster analysis performed on the 13 variables resulted in the identification of four types of teachers based on the characteristics of the course group where the teaching took place. According to the analysis of variance, the four typologies differed significantly in the averages of each of the previously mentioned variables (p < 0.001). According to Dunnett’s T3 multiple comparison tests, eight of the variables, on average, were significantly different in all the groups (p < 0.01). See Table 1.

Table 1. Mean scores (SD) of the variables by groups obtained by hierarchical cluster analysis.

Table 2 showed standardized mean scores of the variables by the group.

Table 2. Standardized mean scores, in standard deviations, of the variables by the group.

Group 1: Average perception of the teaching quality.

Teachers who worked with the largest numbers of students, 55 students on average, the largest number of students attending the course for the first time, 39, with 58.1% passing and an average grade in the courses of 3.8 (both variables with 0.15 SD under the general mean), an academic performance that means failing a course. The student’s perception of the teaching quality was above the general mean. Its lowest value corresponds to the dimension of learning achieved (0.11 SD of the general mean) and Interpersonal relation (0.14 SD of the general mean). See Tables 1, 2.

Group 2: Very good perception of the teaching quality.

Teachers who worked with the smallest group of students, 14.6 students on average, the fewest attending the subject for the first time (7.4%), with the highest pass percentage (80.3%), and with the highest pass average of the courses (4.4), which means an adequate academic performance considering they are STEM subjects. The perception of the teaching quality was very good, the highest assessment of all the groups. In all the dimensions it was above the general mean, with its highest values being in the dimensions of learning achieved (1.07 SD), interpersonal relations (1.05 SD), and Comprehensive education (1.05 SD). See Tables 1, 2.

Group 3: Poor perception of the teaching quality.

Teachers who worked with 21 students on average, only 13.5% of the students attending the course for the first time, with 70.7% passing and an average grade in the courses of 4.1, the second-best grade. The student’s perception of the teaching quality was poor, in all the dimensions it was below the general mean, with its values being lower in the dimensions of organization and responsibility (0.29 SD below the overall average), evaluations, the students perceived the evaluations as inconsistent with the teaching, (0.19 SD below the overall average) and instruction and clarity (0.15 SD below the overall average). See Tables 1, 2.

Group 4: Very poor perception of the teaching quality.

Teachers who worked with 41 students on average, only 30% of the students attending the course for the first time, with 38.8% passing and an average grade in the courses of 3.2, the poorest academic performance. The student’s perception of the teaching quality, according to the teaching evaluation survey, was very low. In all the dimensions, it was below the general mean, resulting in an SD of 1.55 or more, with its lowest values being in the dimensions of interpersonal relation (1.7 SD), evaluations (1.69 SD), and comprehensive education (1.67 SD). See Tables 1, 2.

Qualitative Results

The results of the content analysis show, first, that few students use the comments to express open ideas. About the quality of these comments, they vary in content and length. There were comments which contained only: “good teacher” or “excellent teacher”. By contrast, in other cases they were up to 15 lines long, addressing different course-related topics or describing complex situations (Table 3).

Table 3. Characterization of the comments by discipline, 2016 and 2017.

Types of Coding

The content analysis shows us that topics addressed in the students’ comments concentrated on five central dimensions: Pedagogical management; Classroom climate management; Teaching dimension; Evaluation skills and Student dimension: metacognition. Within each of these dimensions, there was a series of contents addressed, which are illustrated in Figure 1.

Figure 1. Analytical dimensions.

The Dimension Associated With Teaching Management

This dimension addresses mainly students’ perceptions of their teachers in the form of descriptive comments. It refers to perceptions of specific didactic skills, in this case, teaching management (Villarroel and Bruna, 2017). In the area of teaching management, we refer to teaching practices, for example, practices associated with planning, designing, and organizing the course: objectives, contents, scheduling, etc.

The Dimension Associated With Classroom Climate Management

Regarding classroom management or classroom climate, we refer to teaching practices connected to leadership and group management, relationship with the teacher, willingness to understand the student, management of an atmosphere of trust, among others. Practices include, for example, designing spaces for participation in the course and spaces of trust in the classroom that facilitate learning. Texts emerged in the comments recounting the teacher’s student group management, the negative and positive treatment of students, discriminatory practices of teachers toward students, finding discriminatory practices mainly due to gender and performance, and less referenced but present, sexual harassment and descriptions of teacher absences. In the subdimension Instructor descriptions, the appreciation and value that students placed on the teacher’s knowledge of their discipline, and how the students appreciated the general didactic tools that the teacher used appear significant.

Teaching Dimension

This dimension highlights perceptions about a specific teaching skill: didactic skills, use of teaching methods (Villarroel and Bruna, 2017). About these practices, we identified elements related to the organization of contents, delivery of comprehensible instructions, the use of examples, and application of knowledge. From the area of communication, feedback to the teaching-learning process, of the evaluations and for the evaluations, among the most outstanding practices. Among other perceptions that emerged and stood out is the teacher’s motivation to teach the class. Additionally, there are references to the student’s perception of the teaching pace and the lack of interest that some teachers show in teaching.

Evaluations Dimension

This dimension reveals perceptions of the teacher’s use of various, pertinent, and demanding strategies (Villarroel and Bruna, 2017). This dimension was quite limited but very present. It mainly addressed the contents that included evaluations and the adequate level of difficulty of these evaluations.

Student Dimension

This dimension is more analytical than the previous ones. On the one hand, it sometimes refers to students’ capacity for self-criticism and analytical capacity about their own study experiences: observations about themselves or their classmates in terms of the presence or absence of prior knowledge brought from high school and the knowledge gained before the course mentioned. Related to this, there is only one course that refers to camaraderie to correct the differences in students’ previous knowledge. There are also more isolated references that appear a couple of times about the fear of dropping out. The expectations appear as very clear elements in the observations; they are constructed in conditional or future simple, and mainly express desires of a different way to function in the classroom, the types of evaluations, and the teacher’s role. Specific and general wishes also appear. One final element in this dimension is the expression of emotions and feelings of victimization and aggression by the teachers, especially about asking for concepts not understood or not known.

Most Referenced Codes

For the analysis of the most relevant codes among the twelve analysis units, we hierarchically considered the five most frequent codes for each unit analyzed. We compared these codes among the 12 units to examine which of them shared relevance. The analysis yielded the five most frequently mentioned groups: (1). Pedagogical management, classroom climate management, methodology, and didactics, presented a relevance in eleven of the twelve analysis units; (2). Instruction clarity, which presented relevance in nine of the twelve analysis units; (3). Feedback to the teaching-learning process and treatment of students, with relevance in eight of the twelve analysis units; (4). The codes student’s expectation lessons functioning and student learning experience, with relevance in four of the twelve analysis units; (5). Finally, discipline knowledge and classroom climate were relevant in three of the twelve analysis units.

The comments about the teachers concerning classroom climate management contained references that described the teachers with positive and negative adjectives. It is worth noting that of approximately 400 references in this code, only 4% described the teachers negatively. Most of these comments considered positive were comprised of one adjective or its superlative to describe the teachers: “good teacher,” “very good teacher,” or “excellent teacher”. Sometimes these descriptions included more elements that enabled a deeper explanation of how the students perceived their teachers: “A very approachable, empathic teacher who gives good advice. In general terms a very nice teacher,” (comment Chemistry 2016-1) or “Very good professional, likable, knows how to reach the students, funny” (Mathematics 2016-1). The positive comments, although they are mainly not very thoughtful or detailed, referred to aspects of character: friendliness, empathy, easy-going, “nice person” or “cool” in slang, referring to someone of good human quality, polite, honest, and fair.

However, the negative references were more detailed, with comments alluding to classroom climate management, incorporating elements of teaching competencies associated with pedagogical management, as we can see in these examples. The students perceived that not every teacher was trained to teach in initial courses: “I think he is very inadequate, especially for first-year students.” (Mathematics 2016-1). Likewise, when the teacher did not consider the students’ previous knowledge and reacts unexpectedly: “Bad teacher, assumes that we already know the material and because he did not cover it and people ask a lot of questions, he gets angry and laughs.” (Physics 2017-2). Associated with this, we also see that the negative comments addressed the teachers’ arrogance and their inappropriate comments in class: “He jokes about his money, his salary or the great trips he has taken; I would define it as a little over the top” (Physics 2016-1). Another example about the comments made in class refers to a homophobic teacher who made negative comments in a course with openly homosexual students: “I consider that as a teacher, he should be more professional and leave his points of view referring to a certain type of people outside the classroom” (Physics 2016-2). Finally, from pedagogical management, the lack of planning for the semester, the slowness, and lack of dynamism in the teacher receive negative comments.

Continuing with classroom climate management, the results show that the treatment of students is generally quite good; however, a derisory and reproachful treatment is frequently observed of those students who asked questions about basic contents. This treatment refers to recrimination from the teachers for asking questions about contents the students should know given that they had passed the previous subject. On the other hand, this recrimination became insulting and offensive: “Generally, they do not build trust in terms of asking questions because they are going to make fun” (Physics 2017-2). Another detailed example gives us an impression of this treatment as soon as the students confronted the teacher about his pedagogical management, for not returning the results of evaluations promptly: “After this changes for the worse in his attitude, he not only treated his students as lazy every time they asked questions but me in particular. I felt humiliated with the teacher’s treatment once I asked him something, he speaks to the class and uses me as an example, so they do not follow my study method, so they can pass the course, something that seems to be unacceptable. After that encounter I took my things and left his class” (Mathematics 2017-1). There are even accounts that reveal homophobic or misogynist treatment by the teachers. Here is an example: “[the professor has] a homophobic and contemptuous attitude with students in the course, very unequal treatment of the students. More than one classmate has noticed this situation. Ingratiating attitudes with some female students in the course and ignoring others” (Chemistry 2017-1).

Continuing with classroom climate management, some mathematics teachers, for example, had a very good evaluation in terms of the use of humor as a way to ease the tension, generating a safe space with trust, as this comment indicates: “The teacher creates comfortable and didactic spaces in class, which help a lot for learning what he teaches” (Mathematics 2016-2). Also, this example: “Sometimes he tells anecdotes or stories in class which makes the learning period less stressful” (Mathematics 2017-1). In addition, comments note that they tend toward this protected atmosphere of trust, which is related to dynamism, encouraging the student. As actions that prevent spaces of comfort and confidence in the classroom, these include teacher behaviors related to inflexibility regarding lateness, the use of shouting, misogyny, or disparaging words when the students do not know something or ask something that it is assumed they should know. An example from chemistry: “Impossible to ask questions in every class, his arrogant and unfeeling attitude makes for a very tedious environment to work with him” (Chemistry 2017-1).

On the teaching dimension: didactic skills, we noted that the information is not detailed. The comments referred to the teacher as having an excellent or very poor way of teaching. In addition, the presence of very entertaining and educational classes was mentioned, but there was little development about what makes the classes entertaining or educational. The same occurs with the references that criticized monotonous or boring classes. However, the monotony and lack of dynamism appear associated in other references with the teaching of theoretical concepts and definitions as well as practices where the teacher copies directly from a book or study plan and transfers it to the whiteboard. This monotony is associated with classes, in which only one type of activity is performed. In the worst case, even courses conducted in the same way through the entire lecture period: “his classes are only to read the PPT (PowerPoint), it’s almost the same as if I stayed home and read the material and studied on the Internet, without needing to go to classes…” (Chemistry 2016-1).

The most detailed references show us that, generally, for all the analysis units the classes perceived as methodologically and didactically positive were those where: “The classes are complete: space for explanation, theory, and exercises” (Mathematics 2016-1). It is also valued as positive that the theoretical explanations were based on daily phenomena and related to the program for which they were being prepared: “Teacher illustrates how mathematics act in the description of phenomena related to the program; for example, the analysis of a signal, phasors, Fourier transform of a Gaussian, among others. He explains natural phenomena from a mathematical point of view. He brings a thermos with tea for the students on cold days!” (Mathematics 2017-1). In this last respect, elements associated with classroom climate management are emphasized again, underscoring actions by the teacher that make it possible to bond with their students.

Other didactic strategies identified are visual, such as the use of colored markers and memes. Auditory strategies: funny stories and anecdotes to break the monotony in theory classes, encouraging student participation, sending them to the whiteboard to solve exercises, or using group work. Of the motivational pedagogical practices, the following stand out: “I think it is a good idea to send emails to the students, in general, so they care about the course” (Physics 2016-2).

Instruction clarity was a widely referenced code too. The teachers’ skill at making the students understand abstract content, and for many, content they were learning for the first time, was valued positively in this code. In this sense, clarity was a broad concept that referred to the ability to explain, illustrate in different ways, and make complex subjects clear to a heterogeneous class. Some students came from secondary schools where they did not cover contents that were then considered “basic” in the courses analyzed, whereas other students came with solid knowledge. Clarity was also closely related to the teacher’s ability for feedback. Feedback was identified as the first communicative context: question and answer. And thus, that feedback was constant, fluid, and in a space of trust: “I wish there were more teachers like this, who were always open to questions from their students, that they don’t make fun of the student for simply not knowing” (Physics, 2017-2). In this comment elements associated with classroom, climate management are highlighted again, this time emphasizing values such as respect for the students.

The student dimension: metacognition was strongly intertwined with descriptions of the methodology and didactics used by the teachers. Although this code refers to the students’ wishes and not to what happened in the classroom, it reveals, to a certain extent, what the students in STEM courses experience and those aspects to be expected and those absent in their training in STEM courses. However, the expectations were related to the six dimensions analyzed: pedagogical management; classroom climate management; teaching dimension; evaluation skills; student dimension: metacognition and contextual dimension. They were different from these codes for being stated in conditional and future simple conjugations. In some cases, these comments about expectations were aimed at an ideal reader who could take charge of the demand raised. We see in this example: “It would be better to prepare [the teachers] to teach better or just the teachers who teach in those laboratories, also that, if they want better results in the use of Matlab, they should increase those lab hours, since I don’t think that with one hour, the expected learning is achieved (obviously counting the extracurricular hours), I hope that they take the subject of the labs into account although it was not bad for me […]” (Mathematics 2016-1).

Discussion

The first observation of the results of student academic context variables demonstrates that STEM disciplines in this university have very low performance results, ranging from 3.2 to 4.4 in the four groups in the cluster. The best performance was expressed in group 2, which was also the smallest in terms of students (14.6 on average) and only 7.4% were attending the course for the first time, which suggests that this group mainly failed or dropped the course. In addition, of the four groups, groups 1 and 4 obtained on average a failing grade, and these groups were also the largest, where 60% had attended the course for the second time, which suggests serious problems and challenges for students to achieve the learning outcomes. From this perspective, these findings of high failure rates, which would also imply dropout rates in these disciplines, are indicators of low educational quality consistent with the literature (Paterson, 2017; Dewberry and Jackson, 2018).

Next, Table 4 shows the results summary based on the organization of the dimensions. The quantitative results demonstrated the variables that were most frequently repeated according to the highest and lowest scores in the different groups revealed by the cluster. The qualitative results provide five groups of codes of greatest frequency in the twelve analysis units referring to the students’ comments. These analyses performed in parallel appeared in all the established analytical dimensions, although with differing importance.

Table 4. Result-synthesis mixed-methods analysis.

One aspect that highlights the situation of poor academic performance in the different groups and that is accompanied by very low scores in the analytical dimensions of pedagogical management, classroom climate management, and teaching dimension is that the results of the content analysis describe the students’ scarce participation in making comments (approximately 8% for both years) and how short those comments are. This has important civic implications, especially considering the Chilean political process experienced in the last few decades, characterized by a deep social malaise (Durán, 2018), associated with the segmentation of the education system and social segregation (Donoso, 2013; Murillo and Martínez-Garrido, 2017). The main scenario of making demands has been through the occupation of public spaces in organized student marches, also characterized by degrees of violence (Montoni Rios, 2019). However, university spaces have also been “occupied” for demands, with student organizations standardizing strategies for occupying university buildings (prohibiting entrance by staff and students and interrupting activities), protests in the university setting, and systematic student strikes for a semester or a year. This suggests that not only can the quality of education be negatively affected, but also that civic participation by university students is conceived as being limited to standardized practices institutionalized in recent decades, i.e., prolonged student strikes and mechanisms of coercive dialogue. Thus, when students have the opportunity to participate in teaching evaluations, with the protection of anonymity, this participation is the lowest and with very little content. The political participation of young people in the electoral system is very limited (Contreras-Aguirre and Morales-Quiroga, 2014), which is a trend at the moment. In this sense, it becomes necessary to provide spaces for reflection in academic departments and, as the literature indicates, in the relevant figure of the department head (Coleman et al., 2019), because a deep understanding of effective pedagogy involves developing proactive participation in the students, as part of their civic education, with meaning and substance that enables feedback on teaching practices and the student’s conduct in their active role of learning.

The qualitative and quantitative results, from the analytical dimension of teaching management, showed, with high and low scores, that students highlight aspects of teaching related to the organization and responsibility of the teaching staff, which stood out in the textuality of pedagogical management and discipline knowledge, and the comments were largely positive, for example about the teacher’s previous preparation of the classes or activities in the module, or the development of classes according to the planning presented in the syllabus at the beginning of the semester (Gess-Newsome, 2015; Auerbach and Andrews, 2018; Sabah and Du, 2018; Borda et al., 2020).

From the dimension of classroom climate management, the results demonstrate that interpersonal relations play a very important role in teaching, as revealed by three of the four groups and with a high frequency in the comments. This dimension emerges with a negative perception in Groups 1 and 4, just as in Sabah and Du (2018), and positive perception in Group 2, described with relevance in aspects associated with the interest the teacher shows in their students, climate management, perception of the climate and the treatment of the students. This suggests its importance as a key component in teaching competencies, because Interpersonal Relations are part of student-centered teaching (Brown et al., 2014, cited in Dennin et al., 2018; Kober, 2015; Trenshaw et al., 2016), and in positive learning environments. In this respect, the socio-emotional competencies of teachers contribute to developing learning environments that promote student participation in class and constant feedback with one paradigm supporting the other, delving more deeply into qualifications that broaden awareness of the issues that affect the classroom culture related to race/ethnic group, lesbian, gay, bisexual, transgender and queer (LGBTQ) status, religious affiliation, ability, socioeconomic status and other social identities that contribute to the disparities in the performance and persistence of STEM (O’Leary et al., 2020).

Comprehensive education emerged in this dimension, with a high and low assessment, mentioned in aspects such as the promotion of attitudes of tolerance in the teacher, social commitment, and respect for diversity. In this respect, the findings repeatedly showed that students often do not feel sufficiently confident to ask because the teachers use resources like irony, ridicule, or scolding in the face of questions they consider obvious or that should not have been asked. Hence, the teacher’s understanding of their students’ socio-educational characteristics acquires importance, especially in the context of social and educational segregation in Chile. This results in students entering university with large differences in skills development and learning outcomes. This is most noticeable in the STEM disciplines, which require great abilities of abstraction and skills in math and science, representing a tremendous teaching challenge even in STEM (Borda et al., 2020). This discrepancy in skills and learning results in high failure and/or dropout rates in the initial years.

As to the teaching dimension, the analyses suggest it is valued by students but generally poorly assessed, especially in aspects of the teacher’s instruction and clarity, where they emphasized elements related to the instruction of the course contents, elements of didactics like answering questions, explaining the content and giving instructions, and others. In the comments, aspects associated with the use of Methodology and didactics showed traditional teaching practices, meaning in those subjects that were practical, a more teacher-centered class predominated. Concerning instruction-clarity, the student evaluation gains greater importance, in particular considering the inequality in learning and skills that students arrive within first cycle subjects. The greatest challenge is for teachers to have and to demonstrate knowledge of the principles of how people learn, which strategies facilitate, for example, a reduction in the gap between students and promote active learning behaviors. This requires that the teacher have well-developed pedagogical content knowledge (Gess-Newsome, 2015) so they can teach active learning in STEM courses, visualizing, among other elements, feedback to the teaching-learning process. In this aspect, the comments reveal little practice of feedback to the teaching-learning process, very far from what the updated literature reports for this important dimension of the teaching-learning process, in which the student assumes an active role (Flodén, 2017; Van der Kleij et al., 2019; Winstone et al., 2021). This also implies a limited implementation of formative evaluation (Sabah and Du, 2018), a relevant strategy to promote learning achievements. Evaluation skills scored positively in group 1, where most of the students were taking the course for the second time and it was the smallest group. We found no qualitative evidence because it has a very low frequency.

From the dimension of evaluation skills, the results demonstrate a poor perception on the part of the students, particularly in groups 3 and 4; however, it is noteworthy that this dimension presented the lowest frequency in the comments, with negative aspects that indicated that the evaluation methods were not adequate, that there is rarely the possibility of feedback to reach the objectives and that the evaluation did not fit the criteria previously indicated by the teacher. It is important to indicate that the poor perception of this dimension in groups 3 and 4 is also consistent with the low and very low perception of teaching quality and the groups with the worst average performance and the highest average number of students.

The qualitative and quantitative results for the student dimension: metacognition is concerning because added to the indicators of poor performances in the courses studied is the negative perception of the learning achieved in three of the four groups, with group 2 being the one that does not score low. However, group 2 is the smallest group and most of the students were taking the course for the second time. By contrast, the other three groups include most of the students, the poorest performances, and many of them have taken the course more than once. Nevertheless, the comments are of low relevance in their frequency.

Conclusion

The main conclusions from the findings suggest that pedagogical content knowledge in STEM teachers must continue to evolve through various teacher training programs and highlight in them the active role of the student. It is proposed that improvement policies should be developed from the faculty level and with the department head as key to producing working groups with STEM objectives. Specifically, we can indicate: a) orienting teacher training with special emphasis on pedagogical content knowledge to promote active learning in STEM disciplines; b) deepening knowledge of the classroom culture and its issues associated with low outcomes to offer students equitable opportunities for academic achievement in STEM; and, c) developing socio-emotional and management competencies for a favorable learning environment for all the students, inclusive classrooms that make it possible to eliminate the burden of academic success being only on the students. In addition, in the training sphere, students should play an active role in the improvement of learning-teaching processes, extending their civic participation in formal spaces of permanent dialogue, that move beyond the current forms of student pressure and toward the synergy of the university community. Finally, it is necessary to improve the instrument of instructor evaluation in general aspects of the teacher’s pedagogical knowledge and, in particular, to incorporate a dimension in the feedback process, but from the most up-to-date theoretical perspectives (communicative and dialogic models) that place the students in an active role.

The findings of this study contribute empirical data to determine profiles of good educational practices in STEM disciplines based on student perceptions in a local context. Methodologically, the mixed design made it possible to deepen knowledge of some dimensions of these teaching practices; moreover, the study contributes with input to orient internal management policies that can improve the quality of STEM education.

Despite contributing to the literature on teaching practices in STEM disciplines, this study has some limitations. First, the analyses were performed with cross-sectional data. Second, the reported data consist of self-reports from the students. Future studies should consider improving the teacher evaluation instrument, the longitudinal measurement, multiple reporters (instructors, students, and university decision-makers), and continue with mixed research designs that can expand on the narratives of the actors in greater detail.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by the Research Ethics Committee of the University. N° 061_18. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

SS-G created the research question, contributed to the methodological design, performed the quantitative data analysis, and generated the results. MB-S conducted a bibliographic search, designed the theoretical framework, and contributed to the discussion. JS-T contributed to the methodological design, performed the qualitative data analysis, and generated the results. CT contributed the theoretical framework and the discussion. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the Dirección de Investigación, Universidad de La Frontera, Proyecto IF18-0013.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors wish to thank the Vicerrectoría de Pregrado de la Universidad de La Frontera for providing access to the corresponding database. They also wish to thank the Cátedra Unesco Bienestar de la Niñez y Adolescencia, Educación y Sociedad.

References