Introduction

In several educational documents, research and design activities are identified as important focus points in K12 Science, Technology, Engineering, and Mathematics (STEM) education (NRC Framework 2012; NGSS 2013; Platform Onderwijs2032 2016). Two integrated STEM subjects that focus on research and design skills were introduced in The Netherlands: O&O (the Dutch abbreviation for “Onderzoeken en Ontwerpen,” that is, “Research and Design”) in 2004, and NLT (nature, life, and technology) in 2007. Both subjects are elective and entirely project-based. O&O is taught for 4–6 hours a week in grades 7–12 (ages 12–18); the projects run for about 10 weeks in the lower grades, and in the upper grades, students choose projects themselves which last for 20 or 40 weeks. NLT is taught for 3–4 hours a week in grades 10–12 (ages 16–18), and each project takes about 8–10 weeks to complete. The subjects’ main difference is that NLT is more research and science oriented, and O&O has an equal amount of research projects and design projects. Each project revolves around two or more STEM domains connected in authentic real-world contexts and bound by STEM practices and characteristics that fit the description of integrated STEM education (Kelley and Knowles 2016). O&O and NLT are unique types of subjects that employ research and design projects in STEM all year through, instead of embedding these projects in the regular science curriculum in the form of short-term projects (Johnson 2013; Van Breukelen et al. 2017).

Teachers play a big part in shaping such new subjects in the curriculum—they are the biggest influence on whether the new approach is implemented successfully into practice (Van Driel et al. 2001, 2005). However, teachers of integrated STEM subjects are not specifically educated to teach all the different kinds of STEM projects that the O&O or NLT subjects entail (Honey et al. 2014). Teachers of NLT are qualified to teach one single science subject (biology, physics, chemistry, mathematics or geography), and do not participate in specific professional learning for NLT. O&O teachers can be teachers of any subject (from physics to history to languages). They receive basic training in six courses—each half a day or a day long, with intermediate assignments and return days—on how to supervise interdisciplinary research projects and design, on how to assess these projects, and on how to develop projects in collaboration with local companies using authentic problems. Thus, it is often the case that O&O and NLT teachers are not content experts in every project, but rather act as coaches who supervise students who conduct these integrated STEM projects.

In this paper, the term STEM teachers refer to teachers of integrated STEM subjects (like O&O and NLT). Most STEM teachers are not specifically educated to supervise research and design in multiple contexts, and not much is yet known of these teachers’ outlook and feelings of competence when doing research projects and design projects with their students. To understand their outlooks on supervising research projects and design projects in the classroom, we investigated the attitudes present in two different populations of STEM teachers (O&O and NLT teachers) who supervise research projects and design projects conducted by their students. Teachers’ variables, like a teacher’s attitude, are important in shaping student attitudes and in determining whether the introduction of new integrated STEM subjects will be successful (Denessen et al. 2015; Osborne et al. 2003; Van Driel et al. 2001). Results from this study may uncover possible problems that teachers experience when supervising research or design and may show differences between the two different STEM teacher populations.

Our research questions were:

  1. 1.

    What are the general attitudes of STEM teachers towards supervising research projects and towards supervising design projects?

  2. 2.

    What are the differences in attitude between and within two different types of STEM teacher populations, that is, teachers of O&O and teachers of NLT?

  3. 3.

    What are the differences in attitude between and within O&O teachers with different disciplinary backgrounds (science versus non-science)?

Theoretical Framework

Teaching STEM

Educational policies like the Next-Generation Science Standards (NGSS 2013) place emphasis on providing stronger connections between STEM disciplines because “most global challenges concerning energy, health, and the environment (e.g., climate change, sustainability) require an interdisciplinary (and frequently, international) perspective involving mathematics, science, and technology” (Shernoff et al. 2017 p. 2). With integrated STEM, educators try to combine science, technology, engineering, and mathematics disciplines into one subject. It should be clarified that STEM learning can involve multiple subjects and need not involve all four STEM disciplines (Stohlmann et al. 2012). However, limited research is available on how teachers could instruct integrated STEM since it is a relatively new field of education (Stohlmann et al. 2012), and few teachers are specifically trained to teach integrated STEM as most Dutch secondary school teachers have only received education in one discipline (Honey et al. 2014). Shernoff et al. (2017) state that this causes concern over the quality of education and teacher skills in STEM. Thus, the existing literature implies a need for greater teacher education in relation to teaching integrated STEM subjects.

Asking teachers to teach in STEM areas other than their own discipline creates new challenges and knowledge gaps (Stinson et al. 2009). Shernoff et al. (2017) found that teachers stated that “they did not know how to effectively integrate the STEM areas”, and that “their lack of understanding of how to teach in integrated ways was strongly related to students’ lack of understanding or lack of motivation to learn in different ways” (p. 8). Teachers expressed that a shift in mindset was needed: teachers and students needed to get used to the idea that the teacher’s role was not to give the students the correct answer to the given tasks (Shernoff et al. 2017). Teachers of integrated STEM also emphasize the importance of support in areas outside their expertise, time to prepare, implement and evaluate a project, or to work with colleagues and resources (Eijkelhof and Krüger 2009; Shernoff et al. 2017).

Over the last few decades, the technology and engineering components of STEM have been given little attention in schools compared to science and mathematics (Hoachlander and Yanofsky 2011). This seems to be changing slowly. The engineering design process, in which students solve a problem by developing products or services in a systematic and iterative way (De Vries et al. 2005), is becoming more important in STEM education curricula because it has the potential to enhance problem solving in real-world science and mathematics problems (Shernoff et al. 2017; Stohlmann et al. 2012) and can act as the “glue” that meaningfully integrates STEM disciplines in K-12 education (Moore et al. 2014a; Moore et al. 2014b). However, very few K-12 teachers are actually trained to teach the engineering design process.

Previous studies provide empirical evidence for the effectiveness of the design process in facilitating the integration of concepts from multiple STEM areas (Estapa and Tank 2017; Guzey et al. 2016), and for the influence of design activities on positive attitudes towards STEM careers and skills like problem solving, creativity, communication, and teamwork (e.g., Glancy et al. 2014; Guzey et al. 2016; Moore et al. 2014b). These findings also touch upon the discussion of whether integrated STEM should focus on the learning of scientific concepts, the learning of skills to be able to engage in scientific and engineering processes, or both. In their definition of STEM education, Kelley and Knowles (2016) place emphasis on content learning in two or more STEM areas and on the importance of enhancing student concept learning. Johnson (2013) describes integrated STEM as “an instructional approach, which integrates the teaching of science and mathematics disciplines through the infusion of the practices of scientific inquiry, technological and engineering design, mathematical analysis, and 21st century interdisciplinary themes and skills (www.p21.org)”. She seems to place emphasis on the inquiry process, the engineering design process, and twenty-first century skills like critical thinking, problem solving, collaboration, and information literacy to teach science and mathematics. Bybee (2010) describes STEM literacy as “the conceptual understandings and procedural skills and abilities for individuals to address STEM-related personal, social, and global issues”, placing emphasis on both conceptual knowledge and procedural skills, like inquiry. Educational documents, moreover, often place emphasis on increasing student knowledge about career opportunities in STEM (NRC 2012). It seems that ideally, teaching integrated STEM results in both student learning of scientific concepts and student skill development in scientific and engineering processes. The focus on conceptual knowledge versus skills has implications for teachers: a strong focus on student knowledge acquisition might imply that teachers actually need to teach or explain content to their students in relation to a STEM project. A strong focus on student skill development asks for a more student-centered approach, like guiding and supervising students (Henze et al. 2007) who engage in research or engineering processes in STEM projects. It remains debatable how skill-focused approaches ensure that students contextualize these skills and that students acquire the underlying conceptual knowledge required to understand the STEM disciplines.

Teacher Attitudes

As described above, most integrated STEM teachers are originally educated to teach subjects in single disciplines. Implementing a new integrated STEM subject as part of curriculum innovation poses challenges for teachers who are not used to teaching these subjects. They are not yet used to the content of the new subject, as it differs from the content of the subjects teachers usually teach (Stohlmann et al. 2012). Also, they have to get used to other, often project-based and student-centered teaching methods and pedagogical approaches, instead of letting students complete workbook questions after a teacher’s explanation (Henze et al. 2007). This makes new integrated STEM subjects, like the Dutch subjects O&O and NLT, potentially more difficult subjects to teach. Research indeed shows that teachers from the separate disciplines of science, technology, and mathematics all felt uncomfortable at some point while implementing a new integrated STEM subject (Stohlmann et al. 2012). The degree to which teachers were passionate to continue to develop as a teacher of a new integrated STEM subject decreased their discomfort (Stohlmann et al. 2012). From previous research, we know that teachers react differently towards curriculum innovations and develop different types of knowledge for teaching (for example content-oriented versus skills-oriented) (Cohen and Yarden 2009; Henze et al. 2008). Instructionally effective teachers are often more positive and receptive towards curriculum innovations than less effective teachers, possibly because highly efficient teachers have high personal self-efficacy, feel confident about their teaching abilities, and like teaching (Guskey 1988). The attitudes of teachers will shape their interpretations of newly introduced subjects in the curriculum (Jones and Legon 2014).

Teachers’ attitudes, whether positive or negative, can influence student attitudes (Denessen et al. 2015). Measuring teacher attitude is important because students’ attitudes towards a subject are shaped by observing teachers’ comments and enjoyment when teaching about a topic (Frenzel et al. 2009). By attitude, we mean the personal outlook of an individual on a certain subject, which is shaped by one’s knowledge, values, feelings, motivation, and self-esteem (Kind et al. 2007; Van Aalderen-Smeets et al. 2012; ). Teachers’ attitudes are known to be related to teaching effectiveness and choice of instructional strategies (Ernest 1989; Guskey 1988; Jones and Legon 2014), and thus influence the classroom practice of a subject. When teachers hold negative attitudes or anxiety towards the subject they teach, for example math anxiety, they can pass this anxiety on to their students (Geist 2010). Conversely, positive teacher attitudes towards mathematics relate to positive student attitudes and student performance in mathematics (Mensah et al. 2013). Hence, research shows that the teacher variables, such as attitude, are the most significant factor determining student attitude towards a subject, instead of curriculum variables (Osborne et al. 2003).

Attitude has been described as having three components: a cognitive, an affective, and a behavioral component (Eagly and Chaiken 1993). A distinction can be made between one’s personal and one’s professional attitude (Van Aalderen-Smeets et al. 2012). A personal attitude, for example towards science, refers to the attitude of the individual, independent of their profession, and includes for example one’s interest in reading science magazines in their spare time. A professional attitude, in the case of this study, of secondary school STEM teachers, involves beliefs and feelings they have towards teaching STEM projects within the school context. Van Aalderen-Smeets and Walma van der Molen (2013) constructed a Dimensions of Attitude towards Science (DAS) questionnaire based on their framework for attitude (Van Aalderen-Smeets et al. 2012). The DAS was developed in the context of Dutch elementary school teachers teaching science, including questions about teachers’ personal and professional attitude. As we are interested only in STEM teachers’ attitudes towards supervising research and design projects in a school context, we adopted the framework of Van Aalderen-Smeets et al. (2012) and based our questionnaire on the professional attitude section in the DAS and on the corresponding theoretical model. Another reason for the choice of this model is its inclusion of the construct of self-efficacy.

Van Aalderen-Smeets et al. (2012) adapted the traditional tripartite attitude model consisting of the components cognition, affect, and behavior (Eagly and Chaiken 1993). They added an additional component of perceived control (Fig. 1), consisting of the subcategories Self-efficacy and Context Dependency. Self-efficacy is the belief in one’s capabilities to perform on a certain task (in our case supervising research and design projects) and is informed by one’s prior experiences such as successes and failures, and by feedback (Bandura 1997; Jones and Legon 2014). Self-efficacy has been shown to be correlated with teachers’ attitudes, among other factors as prior knowledge and experiences (Jones and Legon 2014), and is also a predictor for teacher behavior and the success of educational reform (Jones and Legon 2014). Context Dependency is the beliefs and feelings teachers have about the influence of external factors on their teaching, for example the influence of available time, support, and teaching materials on their lessons (Van Aalderen-Smeets et al. 2012).

Van Aalderen-Smeets et al. (2012) also divide the cognitive and affective components of the attitude model into different subcategories in their model. The Cognitive Beliefs component consists of the subcategories perceived relevance, perceived difficulty, and gender beliefs. In the context of professional attitude, perceived relevance refers to the importance that a teacher assigns to teaching a topic, stating for example, “It is important that students learn to carry out research and design projects”. Perceived difficulty refers to one’s belief regarding the general difficulty of a topic (in our case, supervising research or design activities), and is a predictor to most behavioral intentions and behavior (Trafimow et al. 2002). Gender beliefs refers to the beliefs that teachers have about the role of gender in teaching or learning a certain topic. The Affective States component consists of the subcategories enjoyment and anxiety. Enjoyment refers to positive emotions, for example, enthusiasm, when teaching a topic (in our case, supervising research or design projects). Anxiety refers to negative emotions, for example, feeling nervous, when supervising research or design projects.

Fig. 1
figure 1

Theoretical framework for attitude towards (teaching) science. Adopted from Van Aalderen-Smeets et al. (2012), p. 176)

Full size image

Research Goal

This study aims to examine the attitudes of two different STEM teacher populations (from the subjects O&O and NLT) towards supervising research and design projects. This study differs from other studies that primarily focus on teachers’ attitudes towards single science subjects, especially mathematics (Ernest 1989; Mensah et al. 2013), or science and technology in general, especiallyparticularly in primary school (Palmer 2004; Tosun 2000; Van Aalderen-Smeets and Walma van der Molen 2013). The subject O&O can in some cases differ substantially from teachers’ original subjects because teachers in languages, art, or history can also supervise projects in this STEM-oriented subject. Thus, O&O teachers of these non-science disciplines are perhaps comparable to primary school teachers who teach science. Primary school teachers often have negative attitudes and experience anxiety when teaching science (Van Aalderen-Smeets et al. 2012). We might expect that this could also be the case for the non-science teachers who teach O&O. However, O&O teachers often volunteer themselves to teach this subject, whereas primary teachers are obliged to also teach science to their students. Teacher autonomy and opportunity to make choices themselves is positively associated with teacher engagement and job satisfaction (Skaalvik and Skaalvik 2014). Based on the study design and the theoretical background, we expected to find some differences between O&O and NLT teachers as these teacher populations vary. We expected O&O teachers to have more positive attitudes than NLT teachers supervising design projects, and NLT teachers to have more positive attitudes than O&O teachers supervising research projects. Overall, we expected quite positive attitudes in both groups of STEM teachers as they have mostly chosen to teach these subjects themselves.

Method

Context: the Dutch STEM Subjects O&O and NLT

The subject O&O was introduced in The Netherlands in 2004 in so-called Technasium schools which are certified to teach this subject. The subject was first introduced in a few local schools as a bottom-up initiative by parents and educators. Fourteen years later, there are 94 certified Technasium schools all over The Netherlands. The subject O&O mainly aims to (1) acquaint students with professions related to STEM and (2) stimulate students to develop skills as competent researchers and designers by letting them handle up-to-date and authentic questions in the science and engineering sector (SLO 2014). To reach these goals, groups of students conduct open research projects and design projects related to STEM. The project topics are provided by local companies and stakeholders who act as “clients.” In the projects, often, multiple STEM domains are involved, for example, a combination of science and engineering, or technology and engineering. This, and the link to authentic practices, makes O&O an integrated STEM subject. In one example of an O&O project, a local petting zoo asks students to develop a game for visitors; in another, a local company asks students to optimize an algae reactor and identify factors that influence algae growth. O&O teachers are not content specialists regarding for example algae growth or game development but rather act as coaches to help the students complete their projects and to help them acquire certain skills like teamwork and project management.

Each project takes about 10 weeks in grades 7–10 (ages 12–16); in grades 11–12 (ages 16–18), students choose projects themselves which last for 20 or 40 weeks. In the lower grades, teachers have written material available to provide their students with steps to complete the project, for example by partial assignments like “the client wants to see five detailed sketches.” In the upper grades, students can choose their projects themselves, and eventually approach clients and stakeholders themselves to create their own project. During the subject O&O, students are assessed on their process (50%) and their product (50%). There are no standardized knowledge tests involved as skill development is the main goal of O&O. Students are expected to integrate conceptual knowledge they learned in thier other subjects into their projects. Teachers assess students’ skills through written project reports, portfolios, meetings with the student groups, presentations, and the final product. Sometimes, when students need information about a certain topic or skill, the teacher can decide to give a workshop, but mostly, the teachers just supervise and coach the students during their projects without giving lectures. Teachers of all subjects can become certified O&O teachers by completing six courses provided by the Technasium foundation: (1) introduction to O&O, (2) writing an O&O project, (3) supervising project management, (4) supervising and coaching of students, (5) assessment and evaluation, and (6) contact with companies and stakeholders. Teachers also have to write and teach an O&O project themselves before getting their certificate. Every year, the Technasium foundation provides a week of additional schooling to help teachers become advanced O&O teachers.

The subject NLT (Dutch abbreviation for Nature, Life and Technology) was introduced in The Netherlands in 2007 as a government initiative. About 220 schools are registered as NLT schools, and 165 schools were members of the NLT association in August 2017. The main aims of NLT are (1) increasing attractiveness of STEM education to increase the flow on to higher STEM education and (2) increase the coherence of the separate STEM subjects (Krüger and Eijkelhof 2010). NLT differs from the traditional single disciplinary subjects such as geography, biology, chemistry, physics, and mathematics in four ways: (1) NLT is interdisciplinary; (2) NLT has a stronger emphasis on career orientation in science and technology fields; (3) NLT integrates technology and science; and (4) NLT shows how mathematics is used within science and technology topics (SLO 2012). Like O&O, NLT is a project- and context-based subject. Students conduct structured projects of 8–10 weeks each related to STEM, such as designing tools for use in the biomedical sciences, or researching the technical aspects of clean water supply. Usually, an NLT project includes some kind of research assignment for the students. Unlike O&O, NLT has an emphasis on developing science concept knowledge in addition to the development of skills. Therefore, the subject NLT sometimes includes knowledge tests to assess students, as well as their written project reports, portfolios, products, and presentations. NLT is interdisciplinary in the sense that the problems in the projects lie “in between” the disciplines of science (such as physics, chemistry, biology, mathematics, computer science, and earth science), for example problems in fields of climate, environment, and ICT (Eijkelhof and Krüger 2009). NLT is an integrated STEM subject as technology and mathematics also play an important role in these interdisciplinary problems, and because students conduct projects linked to authentic contexts. NLT teachers are teachers who are qualified in single science subjects: physics, mathematics, chemistry, biology, and geography. There is no official teacher education or qualification for NLT, but teachers can attend an annual NLT convention which offers short lectures and workshops for overseeing projects. Also, NLT teachers can attend general science teacher professional development courses.

O&O is mainly an elective subject that is taught for 4–6 hours a week in all grades 7 to 12 (ages 12–18) of Technasium schools. Unlike O&O, NLT is only taught in grades 10 to 12 (ages 16–18), sometimes as a mandatory subject but often as an elective, for about 3–4 hours a week. In both NLT and O&O, students conduct research and design projects, althought design projects are more common in O&O than in NLT. In general, teachers have more experience in supervising research projects than in supervising design projects because science and inquiry-based methods often receive more attention in schools than the engineering design process (Hoachlander and Yanofsky 2011). Because teachers can often choose voluntarily to teach the subjects O&O and NLT, it is likely that they also have affinity with supervising research and design projects in integrated STEM, suggesting a default positive attitude. However, if schools face a shortage of O&O or NLT teachers, teachers will be appointed to teach O&O or NLT by the school management.

Participants

We approached O&O and NLT schools for this study by selecting schools from databases on the Technasium and NLT subject websites. We invited O&O and NLT teachers to participate in our study by e-mailing the section heads of departments. Teachers who replied distributed the questionnaires to other teachers in their O&O or NLT department. In total, 234 questionnaires were sent to O&O and NLT teachers, distributed as hardcopies by post to be received by the teacher who acted as our contact person. In total, 147 questionnaires were returned from 55 schools situated all over The Netherlands. We approached a larger number of NLT schools than Technasium schools because in NLT schools, often, only 1 or 2 teachers taught NLT, whereas in Technasium schools, O&O teacher teams were generally larger. We obtained passive informed consent from the teachers via an instruction letter. Ethical approval was obtained from the ethics committee of Leiden University Graduate School of Teaching.

Questionnaires that were less than half completed were excluded from the analysis. We also excluded teachers who taught both the subjects O&O and NLT at the moment of filling in the questionnaire to prevent ambiguity in the results as we aimed to compare O&O and NLT teachers. In total, 78 O&O teachers and 52 NLT teachers were included in further analyses (Table 1). Most NLT teachers had an academic (university) degree in science; this is also one of the requirements for NLT teachers. O&O teachers had various educational degrees, mostly in Higher Vocational Education, which entails more practice-oriented studies (including teacher education), and university. This means that they could have some experience with studying science; however, as we do not know which studies the teachers completed, we cannot make any statements about this influence. Almost all teachers taught another subject besides teaching O&O or NLT. All NLT teachers also taught science subjects, mostly physics, chemistry, and biology. Two NLT teachers also taught history, but always combined with NLT and another science subject. Of the O&O teachers, 12 only taught the subject O&O. Two teachers taught another unspecified subject in addition to O&O, while nine teachers taught a science and a non-science subject in addition to O&O. Forty teachers exclusively taught science subjects in addition to O&O: physics, biology, mathematics, chemistry, public understanding of science, and geography. Because geography teachers are also allowed to teach NLT, in our paper, we characterize geography as a science subject to control the comparison between O&O and NLT teachers. Fifteen O&O teachers exclusively taught non-science subjects besides O&O: history, languages, philosophy, and management and organization. We compared these last two groups of O&O teachers to explore possible differences between teachers with an exclusive science background and teachers with an exclusive non-science background.

Table 1 Basic characteristics of participants
Full size table

Design of the Questionnaire

Our Attitudes towards Supervising Research And Design Activities (ASRADA) questionnaire was based on the Dimensions of Attitude towards Science (DAS) questionnaire (Van Aalderen-Smeets and Walma van der Molen 2013), which has been used in the context of elementary school teachers teaching science. As this questionnaire was already constructed in Dutch, there were no translation issues. We adapted the items of DAS to the context of teachers in secondary school, and their attitudes towards supervising research and design activities, instead of science. For the ASRADA questionnaire, we used the attitude components of Van Aalderen-Smeets et al. (2012) (Fig. 1): Relevance, Difficulty, Enjoyment, Anxiety, Self-efficacy, Context Dependency, and Behavioural Intention. The subcategory Gender beliefs was excluded as gender beliefs were not within the scope of this study. The Behavioural Intention component included items on whether teachers intended to attend professional development courses to learn more about supervising research and design projects, instead of asking them whether they intended to supervise more research and design projects within the subjects O&O or NLT because these subjects already solely consist of research and design projects. The questions within each component were asked twice: once for the topic of supervising research activities and once for supervising design activities. The wording of the items was checked by several teacher educators for clarity and consistency. Items were scored on a 1–5 Likert scale, where 1 = strongly disagree and 5 = strongly agree. The complete ASRADA questionnaire was constructed in Dutch and is available upon request (for example items, see Appendix Table 5).

Analyses

The questionnaires were scanned into the computer and data were converted to an SPSS file. We included partly incomplete questionnaires because some teachers only left a few items unanswered. As a consequence, questionnaires with missing values in a certain category were excluded from analyses regarding that category, causing slightly different numbers of individual questionnaires per analysis.

The ASRADA questionnaire was constructed to include 27 items on attitude towards supervising research activities and 27 items on attitudes towards doing design activities. After exclusion of items that lowered Cronbach’s alpha (α), the ASRADA consisted of 51 items in total: 20 items on attitude towards supervising research activities, 22 items on attitudes towards doing design activities, and 9 items on personal variables. The internal consistency for all subcategories in the attitude scale was determined by calculating Cronbach’s alpha (α) (Table 2). The Cronbach’s alpha for the research component of the attitude scale was 0.76, and 0.85 for the design component, making the instrument of sufficient reliability. Calculations for each category were based on slightly different numbers of individual questionnaires as we decided to include questionnaires with some missing values. Exploratory principal component analyses (PCA) showed that the items sufficiently clustered according to the seven subcategories of the attitude model. An instrument very similar to the ASRADA from a previous study on attitudes of secondary school students towards doing research and design activities (Vossen et al. 2018), which was also based on the DAS, showed a similar clustering of all attitude components with even more participants [n = 1625]. In addition, confirmatory factor analyses suggested that the categories in the questionnaire are quite stable.

Table 2 Cronbach’s alpha for the scales for teacher attitudes towards supervising research and design activities. Total number of teachers was n = 130. α Cronbach’s alpha, M mean, SD standard deviation, SE standard error. Note that due to the algorithm for Cronbach’s alpha, all teachers with missing values were excluded from the analysis of each subcategory
Full size table

We analyzed differences between the O&O teacher group and the NLT teacher group by using a linear regression in which we applied a correction for the nested structure of the data to correct for the extra variance in the data given that teachers in our sample all come from different schools. These analyses were also applied to the data to search for possible differences between O&O teachers with a science background and O&O teachers with a non-science background. To discover whether any differences between their attitudes towards supervising research or design projects existed within the O&O teacher group and within the NLT teacher group, paired-samples t tests were applied. All analyses were performed with IBM SPSS Statistics version 22.

Results

The subheadings in this section correspond to the research questions of this study as stated in the Introduction.

General Attitudes of STEM Teachers Towards Supervising Research and Design Activities

The overall attitude towards supervising research and design activities of all STEM teachers in this study was fairly positive. Teachers scored highest on the subcategories Relevance [see Table 2; research: M = 4.10 | design: M = 3.77], Enjoyment [research: M = 4.06 | design: M = 4.08], and Self-efficacy [research: M = 4.06 | design: M = 3.84] on both components (research and design) of the ASRADA. This means teachers found supervising research or design projects a relevant activity, they enjoyed supervising research and design projects and also perceived high self-efficacy while supervising students doing research or design projects. Relevance of supervising research projects was scored higher by the respondents than the Relevance of supervising design projects. The lowest scoring subcategory was Anxiety [research: M = 1.70 | design: M = 1.79], meaning teachers did not feel anxious while supervising student research or design activities. Teachers scored neutral to slightly positive on the subcategory of Behavioural Intention [research: M = 3.17 | design: M = 3.08], which means that on average, they showed no disinterest, but also no clear intention to participate in teacher professional development courses aimed at supervising research or design activities.

Differences between Two Different Groups of STEM Teachers (O&O and NLT)

In the linear regression for nested data in which we compared the attitudes between O&O and NLT teachers, we found that attitudes towards supervising research activities were similar for both O&O and NLT teachers as we found no significant differences between the subcategories for research. It seemed like O&O teachers were somewhat more positive than NLT teachers towards following professional development courses in supervising research (Behavioural Intention) [O&O: M = 3.31, SD = 0.93 | NLT: M = 2.98, SD = 1.06], but this result was not significant [p = 0.058]. However, some clear differences existed between O&O and NLT teachers regarding their attitudes towards supervising design projects. O&O teachers scored significantly higher [p < 0.01] on the subcategories Enjoyment [M = 4.26, SD = 0.66], Self-efficacy [M = 3.97, SD = 0.70], Context [M = 3.68, SD = 0.68], and Behavioral Intention [M = 3.28, SD = 1.11] than NLT teachers [respectively M = 3.75, SD = 0.88 | M = 3.58, SD = 0.81 | M = 2.81, SD = 0.78 | M = 2.76, SD = 1.08], meaning they enjoyed supervising design projects more, experienced more self-efficacy, experienced better enabling contexts to supervise design projects (like available materials), and were more positive towards participating in professional development courses aimed at supervising design activities than NLT teachers. NLT teachers scored significantly higher [p < 0.01] on the subcategories Difficulty [NLT: M = 3.33, SD = 0.59 | O&O: M = 2.95, SD = 0.71] and Anxiety [NLT: M = 1.99, SD = 0.80 | O&O: M = 1.66, SD = 0.58], which means that they saw supervising design projects as more difficult and experienced more anxiety while supervising design projects than O&O teachers.

Within the two teacher populations, there were also differences between teachers’ attitudes towards supervising research activities and their attitudes towards supervising design activities (Table 3). Results from a paired-samples t test showed that O&O teachers scored significantly higher [p < 0.001] on Difficulty towards supervising research projects [M = 3.31, SD = 0.63] compared to supervising design projects [M = 2.95, SD = 0.71], and significantly higher [p < 0.05] on the subcategories Enjoyment [M = 4.26, SD = 0.66] and enabling Context [M = 3.68, SD = 0.68] for supervising design activities compared to supervising research activities [respectively M = 4.03, SD = 0.75 | M = 3.49, SD = 0.79]. There were no significant differences in the subcategories Anxiety, Self-efficacy, and Behavioral Intention. Within the NLT group, teachers scored significantly higher [p < 0.05] on the subcategories Enjoyment [research: M = 4.09, SD = 0.73 | design: M = 3.75, SD = 0.88], Self-efficacy [research: M = 4.16, SD = 0.63 | design: M = 3.58, SD = 0.81], Context [research: M = 3.28, SD = 0.95 | design: M = 2.81, SD = 0.78], and Behavioral Intention [research: M = 2.98, SD = 1.06 | design: M = 2.76, SD = 1.08] to attend professional development regarding supervising research activities, whereas they scored significantly higher on Anxiety towards supervising design activities [design: M = 1.99, SD = 0.80 | research: M = 1.61, SD = 0.58]. Teachers within both groups scored significantly higher on the subcategory Relevance [O&O: M = 4.13, SD = 0.66 | NLT: M = 4.10, SD = 0.60] regarding the supervision of research projects, in comparison to supervising design activities [O&O: M = 3.83, SD = 0.73 | NLT: M = 3.62, SD = 0.80].

Table 3 Differences in attitudes between supervising research and design activities within the O&O teacher group and within the NLT teacher group. Due to individual missing values n is different for every category. For O&O teachers, n varies between 75 and 78. For NLT teachers, n varies between 50 and 52. Significant p values are indicated in italics
Full size table

Differences Between Science and Non-Science O&O Teachers

Within the group of O&O teachers, there are teachers who, besides O&O, exclusively taught science subjects [n = 40], and teachers who exclusively taught non-science subjects (like history and languages) [n = 15]. When comparing differences between these two teacher groups in a linear regression for nested data, we found a significant difference in the subcategory of Behavioral Intention, despite the low sample sizes. Non-science teachers scored significantly higher [p < 0.05] than science teachers on items stating they would consider joining teacher professional development opportunities in supervising research [non-science: M = 3.77, SD = 0.78 | science: M = 3.09, SD = 0.91] or design projects [non-science: M = 3.80, SD = 1.00 | science: M = 3.03, SD = 1.07].

Paired-samples t tests showed that within the O&O teachers with a science background, teachers scored items on Relevance [M = 4.08, SD = 0.67] and Difficulty [M = 3.32, SD = 0.70] of supervising research projects significantly higher [p < 0.01] compared to Relevance [M = 3.76, SD = 0.73] and Difficulty [M = 2.93, SD = 0.79] of supervising design projects (Table 4). This means that the science teachers viewed supervising research projects as more relevant than design projects, but also thought that supervising research projects is more difficult for teachers in general than supervising design projects. Non-science O&O teachers also scored significantly higher [p < 0.05] on the Difficulty scale for supervising research [research: M = 3.24, SD = 0.68 | design: M = 2.91, SD = 0.66], but the difference between the Relevance of supervising research projects [M = 4.09, SD = 0.71] versus supervising design projects [M = 3.88, SD = 0.81] was not significant [p = 0.228].

Table 4 Differences in attitudes towards doing research and design activities within O&O teachers with a science background and within O&O teachers with a non-science background. For science teachers, total n = 40, however, due to individual missing values, n is different for every category, varying between 37 and 40. Significant p values are indicated in italics
Full size table

Discussion

Teacher experiences, attitudes, and beliefs in integrated STEM subjects have not yet been studied extensively. This study aims to contribute to decreasing this knowledge gap in literature. The subjects O&O and NLT provide us with a unique situation in which we can study two types of STEM-based subjects, instead of shorter STEM-based projects. The instrument that was developed for this study could also contribute to further international studies into teachers’ attitudes in delivery of STEM subjects. The subheadings in this section correspond to the research questions of this study as stated in the “Introduction” section.

General Attitudes of STEM Teachers Towards Supervising Research and Design Activities

Overall, we found that the responding STEM teachers held fairly positive attitudes towards supervising research projects and design projects (research question 1). Previous studies also show that both teachers and students hold positive attitudes towards contemporary teaching methods like inquiry and design-based learning (Ara et al. 2011; Damnjanovic 1999; Savelsbergh et al. 2016). Teachers in The Netherlands can mostly choose whether they would like to teach O&O or NLT, and such voluntary choices and degree of autonomy are positively related to engagement, job satisfaction (Skaalvik and Skaalvik 2014), and perhaps also to attitude.

In general, teachers viewed supervising research projects as a more relevant activity than supervising design. This indicates that teachers in general find it more important that students learn how to do a research project than how to conduct a design project. A previous study found that students in general also rate the relevance of doing research activities higher than doing design activities (Vossen et al. 2018). We know that inquiry, or doing research, has long been a desirable skill for students to acquire (Welch et al. 1981; Crawford 2014), and Hoachlander and Yanofsky (2011) have found that engineering components of STEM (such as design) have been given less attention than science components (like doing research). Another remarkable outcome of this study was that all teachers scored rather high on self-efficacy. The teachers in this study thus had high feelings of competence even though they were not extensively trained to teach STEM subjects. One might expect a lower self-efficacy in teachers who teach a fairly innovative subject, especially in O&O teachers who supervise design projects as not many of them have a background in design themselves however, this was not the case. Previous research also found that teachers may hold exaggeratedly positive self-efficacy towards teaching science even if they had no experience (Settlage et al. 2009). Other studies have found that low-performing people often hold overly favorable views of their abilities, while high-performing people tend to slightly underestimate their abilities; the so-called Dunning-Kruger effect (Dunning 2011; Kruger and Dunning 1999; Schlösser et al. 2013). As Tschannen-Moran and Hoy (2007), p.5) mention: “It is important to note that self-efficacy is a motivational construct based on self-perception of competence rather than actual level of competence.”. Reviewing the correlations in our data between the ASRADA subcategories, the categories Self-efficacy and Enjoyment had the highest correlation. Rather than actual competence, the teachers’ high self-efficacy could also be related to high feelings of enthusiasm as literature has shown that teacher attitude has only a very loose correlation to actual teacher knowledge (Allum et al. 2008).

Differences Between Two Different Groups of STEM Teachers (O&O and NLT)

When comparing attitudes towards supervising research projects and supervising design projects between O&O and NLT teachers, we found no significant difference in their attitudes towards supervising research projects. However, in comparison to O&O teachers, NLT teachers perceived more difficulty when supervising design projects. When comparing the attitudes towards supervising research projects and supervising design projects within O&O and NLT teacher groups, we found that O&O teachers were somewhat more positive towards supervising design projects than towards supervising research projects (except on the subcategory Relevance), and NLT teachers were more positive about supervising research projects than about supervising design projects. It seems that teachers tend to rely on their own backgrounds: NLT teachers are qualified teachers of science subjects, and thus they are more used to teaching scientific research methods instead of supervising design. O&O teachers, on the other hand, are a more diverse group of teachers with experience in both supervising research and design projects because about half of the projects in O&O are design-based, and about half are research-based. O&O teachers, like O&O students in an earlier study (Vossen et al. 2018), appear to find supervising or conducting design activities significantly more enjoyable than research activities. It is possible that teachers and students of O&O perceive designing as an activity that has less to do with content knowledge, and therefore finding it “easier” and more enjoyable. Because most teachers can voluntarily choose to teach O&O or NLT, the subjects might attract different types of teachers. It is also possible that because of their lack of experience with design projects, NLT teachers are more negative about supervising design projects than O&O teachers.

Differences Between Science and Non-Science O&O Teachers

The group of non-science O&O teachers could in a way be compared to primary school teachers as both of these groups have no specific prior experience in teaching STEM. However, in contrast to the low self-efficacy for teaching science in primary school teachers (Tosun 2000), the non-science O&O teachers surprisingly had high feelings of self-efficacy towards supervising research projects and design projects, that were not significantly different from the science teachers. These feelings of high self-efficacy could be related to teacher autonomy: primary school teachers are often obliged to teach science somewhere in their curriculum while most O&O teachers are free to choose whether they want to teach this subject. Even though their self-efficacy was high, the non-science O&O teachers had significantly more interest in attending professional development courses than the science O&O teachers. This could indicate that although they already feel competent and enthusiastic, they acknowledge that their competence could grow by acquiring more knowledge and skills for supervising research projects and design projects. They might also be aware of their non-science background. Interestingly, the science O&O teachers and the NLT teachers scored neutral on their intentions to follow professional development courses. Because of their background in science, science O&O teachers may think they do not need further professionalization or alternatively, they might feel there are already enough suitable courses available for them as there are many options for science teacher professionalization in The Netherlands.

Limitations and Implications

As ours was a quantitative study with a closed questionnaire, it would be interesting to include more information about teachers’ backgrounds and teaching practices in qualitative follow-up studies. In this study, we only had limited information on the teachers’ prior education and their experience with conducting research and design themselves. It would be worth discovering the nature of these teaching and learning experiences, and their influence on the development of teacher attitude and the enacted pedagogies during their O&O or NLT lessons. It could be that the more experience teachers have doing research or design tasks themselves, the more positive their attitudes. As we had no information on which teachers had more in-depth experiences in doing research or design than other teachers, we cannot answer this question. Qualitative follow-up studies should also consider student views on the way they are supervised during these research and design projects. Gender beliefs were not within the scope of this study; however, they can influence the way in which teachers approach students (Shepardson and Pizzini 1992). Therefore, additional research on gender beliefs regarding the execution and supervision of research and design problems would be desirable to give more insight into gender beliefs within STEM teachers and students.

Further research is needed to explore why STEM teachers had such high-self-efficacy scores about supervising research and design projects even when they had received no special education in doing so. Their high self-efficacy might not be related to actual competence (Tschannen-Moran and Hoy 2007), but to high levels of enjoyment. It would be interesting to examine the relationships between self-efficacy, enjoyment, and actual competences further in future research, for example by triangulating teachers’ own self-efficacy with other measures of their effectiveness (such as classroom observations, student outcomes and student perceptions of the quality of teacher supervision), and examining the exact correlation between Enjoyment and Self-efficacy scores. It is, however, a promising result that these STEM teachers have high feelings of self-efficacy as this has been shown to be positively related to teacher perseverance (Bandura 1997; Palmer 2006) and student performance (Ashton and Webb 1986). Teachers’ satisfaction with their choice of profession can also relate to high feelings of self-efficacy (Caprara et al. 2006), and O&O and NLT teachers can indeed mostly voluntarily choose whether they want to teach STEM.

Conclusion

The teachers in this study generally found supervising research projects significantly more relevant than supervising design projects. The explanation for this finding should be examined further. National and international curricula already emphasize the importance of the engineering design process (NGSS 2013; SLO 2015); however, the implementation of design activities in schools might not reflect this. The integration of research and design activities are common practice in some university programs and in the professional world (Sanders and Stappers 2008). STEM teacher education should therefore not only familiarize teachers with supervising research projects, but with design projects as well.

The results of this study indicate that there is a need for additional STEM teacher professional learning development, especially for non-science teachers who are beginning to teach in STEM subjects as well. Since STEM teachers have different backgrounds, it is important that ample time, support, and professional development courses are provided to them (Stohlmann et al. 2012). Teacher professional development is often aimed at the content of STEM projects, but for learning to supervise research and design processes, the pedagogy for supervising such projects should also be emphasized. Also, teachers might need first-hand experiences in carrying out research and design projects themselves as not all O&O and NLT teachers necessarily have done this before during their education or career. Instead of already existing courses for single subjects, courses specifically aimed at integrated STEM could attract more STEM teachers and could enhance their willingness to attend such professional development opportunities.