In the twenty-first century, critical thinking skills (CTS) are becoming one of the most crucial learning activities (Fuad et al., 2017; Kavenuke et al., 2020). CTS support students in making decisions in a specific way during the learning process. When students face a given problem, CTS drive a person to analyze a problem and evaluate possible solutions. In this approach, CTS also offer an opportunity for students to use a reasonable rationale for their thinking, reflecting on the problem, and the potential solution (Ennis, 1993).

Critical thinking (CT) has been defined as a cognitive process involving reasonable reflective thinking to develop a decision based on the problem faced by a person; CTS include a person's ability for higher-order thinking, problem-solving, and metacognition (Ennis, 1989). Furthermore, CT is reasonable reflective thinking focused on a decision that the students believe in or do, which is in the cognitive domain (Ennis, 1993). In contrast, Facione (1990) conceptualized that CTS relate to cognitive ability and affective ability to be a good thinker. CT is a reflective thinking skill involving analyzing, evaluating, or synthesizing relevant information to form an argument to make a decision (Ennis, 1993; Ghanizadeh, 2017). Wechsler et al. (2018) proposed that CTS be required to display tests conducted after the CTS process in the classroom and review student behavior during the learning process.

However, research on CTS has been conducted using single tests to determine students' cognitive ability for CT. For example., in a study by Mutakinati et al. (2018), the students in a junior high school were given a CTS post-test following a science lesson. The results showed that the students had sufficient thinking skills to critique their plan for systematic practice, including constructing a realistic critique on their power of thought. Additionally, a study by Fuad et al. (2017) trained students to study more by using exploratory questions and information about how to develop a hypothesis, assisting students in creating learning based on the students' needs. He gave a post-test to the students to evaluate their CTS.

It is crucial to utilize a learning approach that supports a student's thinking in the learning process (Shaw et al., 2020). CTS can be developed during the learning process using teaching approaches that prompt students to face real-world problems. The teacher can select a teaching approach that pushes students to explore their CT through argumentation to make decisions (Ghanizadeh, 2017). One of the teaching approaches to facilitate CTS is the engineering design process (EDP). In the EDP, reflective thinking is needed to produce a better decision in solving a problem given by the teacher. Yu et al. (2020) investigated the relationship between CTS and the EDP when students design a product. The results indicated that the EDP stages played an essential role in students' understanding of their own CTS. Using EDP student experienced to define a problem, develop the argumentation, and finally make a decision that matches the EDP step (Spector & Ma, 2019; Sulaeman et al., 2021). The EDP implementation with several stages of learning, allows students to define the problem before they make a decision (Arık & Topçu, 2020; Tank et al., 2018).

Research exploring how EDP stages engage CTS is scarce. Additionally, most of the research on CTS has been conducted quantitatively using statistical analyses (e.g., Kavenuke et al., 2020; Mutakinati et al., 2018; Yu et al., 2020). The performance of CTS is also essential to highlight the students' behavior during the utilization of their CT abilities (Ennis, 1993). This study explores the students' CTS in the EDP project. One challenge in this study is the possibility of excessive subjectivity when analyzing CT performance; the authors used the peer review process to reduce this potential concern (Merriam & Tisdell, 2016). Thus, the research questions to guide this study are as follows:

  1. (1)

    To what extent could the EDP support CTS?

  2. (2)

    How does the EDP support a student's CTS in the physics classroom by defining the problem, using argumentation, and developing a solution?

Theoretical Framework

Critical Thinking Skills (CTS)

The urgency of CTS could be traced from the educational theory by Dewey. Experiential learning theory explains the practical learning in enquiry practices (Dewey, 1993). Dewey suggested that the essence of an enquiry is formulated in the experiential learning cycle, which is initiated with the perception of solving a problem and exploration of relevant knowledge to construct a meaningful explanation of the solution in solving the problem (Garrison et al., 2001). This experiential learning cycle is a form of reflective thinking to produce better solutions (Garrison & Arbaugh, 2007). Reflection demands to think critically to identify solutions from a problem (Antonieta et al., 2005). CT is reasonable thinking to develop a decision based on a problem caused (Ennis, 1989). The CT be included a person skill in reflecting a solution of a problem given (Ennis, 1993). CTS need to be identified in both cognitive and affective concepts (Facione, 1990; Shaw et al., 2020). Kavenuke et al. (2020) explained that CTS involves synthesizing, analyzing, and evaluating information to make a cognitive decision and transform it into affective domain performance.

Besides being related to the cognitive domain, CTS is also related to the affective domain. This domain engages students in communication to support their decision through argumentation (Antonieta et al., 2005). Students have an opportunity to criticize using scientific statements in a scientific environment when they are communicating their idea (Farmer & Wilkinson, 2018). CTS begin with a simple experience, such as observing a difference, encountering a problem, or questioning someone's statement, and then leads to an enquiry; then, more complex experiences are encountered, such as interactions through communication in the application of higher-order thinking skills (Spector & Ma, 2019).

Measurement tools have been developed using criteria to describe a person's ability in CT. Ernst and Monroe (2004) analyzed criteria while measuring CTS, such as interpretation, analysis, evaluation, inference, explanation, and self-regulation. Also, CT ability is developed in detail through enquiry, argumentation, and self-regulation (Kabir, 2002; Spector & Ma, 2019). Using argumentation in CT, students can select evidence that supports their decision (Giri & Paily, 2020). Moreover, the measurement of CT was developed based on some of the research and models available. CT could be assessed using an open-ended assessment model, multiple-choice with written justification model, essay testing of critical thinking model, and performance assessment model (Ennis, 1993). In general, the measurement in CTS is given through an experimental study (e.g., Farmer & Wilkinson, 2018; Fuad et al., 2017; Yu et al., 2020). The CTS then can be described in the median data collected statistically to different students' levels, such as low and high (Kim et al., 2013). However, studies to demonstrate the performance of CT in the learning process are lacking, requiring further investigation.

Research in measuring CT through performance tests can be done by applying a learning approach that describes reflective thinking (SEN et al., 2021; Yu et al., 2020). The learning approach shows the cycle learning that includes defining a problem, developing a design solution through scientific argumentation, and deciding. One of the learning approaches that facilitate the cycle model in the classroom is the implementation of the EDP.

Engineering Design Process (EDP)

Engineering is a discipline that solves problems by obeying constraints, using a body of knowledge implemented through science, math, and technological tools (NGSS, 2013; NRC, 2012). Continuing design is a critical aspect in engineering that aims to solve a problem by iterative thinking, being open to the idea of having many possible solutions, along with a meaningful understanding of the integration of science, math, and technological concepts (Guzey et al., 2019; Moore et al., 2014). The design process in engineering is needed to develop collaboration and social communication (Sulaeman et al., 2021; Yazici et al., 2020). Thus, the EDP's goal is to solve a real-world problem with an engineer-designed activity.

The engineering practice is recognized by students during the cycling process, solving a problem in several stages. These stages engage students in identifying a problem, understanding the engineering need, and the opportunity to offer multiple possible solutions (Lottero-perdue et al., 2015; Whitworth & Wheeler, 2017). Students develop a critical understanding of the potentially relevant issues within the problem statement, allowing them to generate the best solution in the engineering classroom (Arık & Topçu, 2020).

The EDP addresses students' abilities to make decisions by defining a problem, developing argumentation, and identifying a solution for the problem (Guzey et al., 2016; Mathis et al., 2017). Therefore, in this study, the EDP stage is a bridge to facilitate CTS. More specifically, the implementation of EDP involves cycling, which starts with defining a problem, learning a scientific concept, planning a solution, trying a solution, and deciding (Tank et al., 2018). Furthermore, each EDP step can facilitate the CTS to develop a solution in the engineering classroom using the investigation through defining a problem, developing argumentation, and making a decision (Ahern et al., 2012).

Methodology

Research Design

A single case study was utilized to explore a student's experience in the EDP classroom in relation to CTS (Yin, 2018). The single case study was selected because of the desire to explore an in-depth EDP based on the criteria of general CTS based on Ernst and Monroe (2004). This study was conducted during the pandemic era (when covid-19 hit in the selected area), so the study was conducted using two different approaches: both online and off-line learning (i.e., blended learning). The authors developed an EDP worksheet that guided individual activities and group activities. Due to the regulations affecting education during the pandemic, the classroom only allowed a maximum of 15 students.

Context of Study

The study was conducted in a physics classroom in one of the high schools, located in one district, in Indonesia. Students never followed the same program during the EDP project, particularly in physics. Through the EDP, the students had to define a problem, learn the physics concepts and the related subjects, develop a solution plan, and make a decision about their solution (Tank et al., 2018). The authors developed a EDP worksheet. This EDP worksheet is an instructional sheet for students to understand the EDP stages in this project (Sulaeman et al., 2021).

The team project addressed one challenge in which students could build a solution based on the given problem. The problem given by the worksheet involved asking students to solve a problem regarding the location of rice fields. The situation addressed the lack of water during the dry season and the amount of water during the rainy season. Figure 1 shows the EDP activities in the classroom, which total 315 min of activities. The project was solved by students individually and in groups to assess the consistency of improving the CTS of the students.

Fig. 1
figure 1

The EDP steps during implementation in the physics classroom

Full size image

Participants

There were 12 students in this study in the tenth grade at the time of data collection. They volunteered their participation, joining the EDP project in both the online and off-line classroom. They also agreed to follow government regulations regarding health protocols and received permission from their parents to participate. The full-time physics teacher identified students' levels in various physics achievements. The demographics of the students are described in Table 1. The level of achievement is divided into three categories: high (student's achievement was more than 75); medium (student's achievement was more than 65 but less than 75); and low (student's achievement was less than 65). The score of 75 was a standard value to grade students' mastery in physics concepts in this school.

Table 1 Participant demographics
Full size table

Data Collection

There were three data sources: text based on the EDP worksheet, the recording of students' group discussion, and the recording of the students' interviews. First, Text was collected based on the students answer from EDP worksheet. EDP worksheet presented students with activities to work on individually, enabling the collection of data regarding students' problem definition and learning. Students then developed an opinion based on the question asked: for example, who has a problem? What is the problem? And who is the user? In the individual work, the students' writing was collected. Second, when student worked in their group, the discussion process was recorded. All the students' communication was transcribed to analyze. The data collection was focused on stages of plan, try, test, and decide. Third, Students were also interviewed to acquire the necessary supporting data on the changes in students' CT abilities (see Online Appendix A).

Data Analysis

The three types of data were analyzed to triangulate strategies for confirming the accuracy of the data (Creswell & Poth, 2016). The authors then identified the stage of the EDP (see Fig. 1) and matched the code of CT. All the data collected were transcribed into text and coded as shown in Table 2. The code of CT was developed using the criteria developed by Ernst and Monroe (2004). All students’ statements on the each EDP stage was read carefully and given a justification based on the CTS criteria. The number of the CTS criteria were calculated in each EDP and presented in Table 3. Those criteria were divided into two levels to express the students' CT abilities based on the development of inductive code in the site (Saldana, 2016). Thus, each author coded the data (see example in Online Appendix B) as a peer examination (Merriam & Tisdell, 2016). When the coding presentation differed between authors, the authors met to negotiate a consensus students’ statement.

Table 2 The coding developed and level of CT
Full size table
Table 3 The students' statements on the relationship between the EDP and criteria of CTS
Full size table

Findings

This research aimed to explore students' CTS through approaches of the EDP in a physics classroom. Our findings discuss the matrix of suitability of CTS aspects and each EDP stage. In addition, the results were organized based on the students' ability to define a problem, provide scientific argumentation, and generate a solution.

Matrix of Suitability of CTS Criteria and EDP Stages

From our analysis of students' worksheets and students' discussions in the physics classroom, the results indicated the justification of CTS criteria in each EDP stages. Table 3 provides code frequencies counts for the students’ statement in the EDP project.

Defining a Problem

In the EDP, defining a solution is the first step for students in the problem-solving process. Students experienced solving a problem with a focus on the necessary part of the situation and the constraints given by the teacher. In the "Define" step, students generally started by interpreting and identifying a problem given by the teacher, highlighting the problem statement and the need for problem-solving processes. Some examples were given that were expressed in the worksheet about the problem given: difficulties of watering in the farm field.

[A1]: Farmers in the western rice field of the village made small wells near their fields with the help of diesel pumps to supply water to their fields. The way for farmers to stop pumping water is by building a dam.

[A1]: The client wants the dam to last a long time with an estimated cost of $2000. It is useful for storing water during the rainy season and supplies water for the dry season so that farmers do not have to pump water anymore.

[A2]: The water supply is low in the irrigation area of the river. The river flow in the area is very small during the dry season, while other factors also influence the depth and width of the river. As a result, the branches of the river sometimes do not reach the rice fields, so the rice plants often lack water.

[A2]: The head of the village wants to make a dam that has a width in total 3 metres, 2.5 metres for storing the water, and ½ metre for anticipating when the water overflows

Students [A1] and [A2] categorized the two statements based on the engineering problem given. They stated the problem, and they clarified a constraint to solve the problem. They did not only state the problem about the lack of watering in the rice field, but also mentioned constraints to solve the problem. Those students were categorized in the high level of interpretation criteria of the CTS. Students provide the problem information and the constrain to solve the problem. On the other hand, the two examples below show students low in CTS.

[U1]: During the dry season, the river flow in the area was very small, so that the water in branches of the river did not reach the rice fields [for watering].

[U2]: There was very little water supply in the irrigation area of the river and the [water] flow rate was very small during the dry season, so [the] availability of water in the river is little during the dry season.

Students [U1] and [U2] were less skilled in interpreting the problem given by the teacher. Even though the students could state the problem in the village, they could not adequately to explain of the constraint when asked to develop a solution.

Students' Argumentation

Students in the EDP classroom also provided an argument when they planned, tried, and tested possible effective solutions. In those steps, students worked as a group to discuss their solution for solving a problem, totalling four students per group. The data shown in this section uses vignettes to show each group's manner of discussion. The discussion in this example shows when students were trying to develop a solution. Students [A1], [S1], and [U1] discussed the possible implementation of one of the physics concepts to build a dam. The student [U1] showed an increase in the level of CTS when joining the discussion section with the group.

Vignette 1

[A1]: I think the materials used must be strong and durable to build the foundation of the dam. We also need the concept of physics using hydrostatic pressure. The lowest foundation is built wide and thicker, while the higher one is like the shape of a cone like this [while demonstrating by hand the shape of cone].

[S1]: Then Pascal's Law also explain the water capacity. During the rainy season, the water does not flow out, or in other words, water can accommodate both in raining and dry season. The water can irrigate the rice fields continuously.

[U1]: Yes, I agree, so we also implement Pascal's Law, and we pay attention to the capacity during the rainy season so that the water doesn't overflow [water spill from the dam].

In Vignette 1, the presented discussion is between the students to plan the creation of the dam. They collected physics-related evidence to emphasize the possible implementation of this real-world problem. They learned the concepts of hydrostatic pressure, serving as a base to solve the problem. Student [A1] demonstrated high CTS because he examined ideas by analyzing the concept of physics. His statement is a clear expression of the CTS of analysis, and student [S1] examined the alternative of concept work in emphasizing the situation by collecting evidence of the amount of water during dry or rainy seasons. She demonstrated high CTS of evaluation criteria. Student [U1] also described and emphasized the solution by drawing a conclusion of the dam being built. He also demonstrated high CTS in terms of the inference criteria.

In the "Test" step, students concluded their problem design and coordinated it with their understanding of the criteria needed. In Vignette 2, the discussion shows students offering further clarification about their design. Students evaluated the criteria of the design based on the problem given.

Vignette 2

[U1]: The dam has a width of 3 metres; it's almost the same as the constraint.

[S1]: Yes, 2.5 metres is added during the rainy season and ½ metres during the dry season.

[A1]: The dam must last a long time with a budget of $2000.

[S1]: Okay, this means, yes, the criteria requested by the client are a dam that has a width of 3 metres and a depth of 2.5 metres during the rainy season, and ½ metre during the dry season, and the dam must last a long time with a budget of $2000." (Writes down the criteria requested by the client)

Students [U1] and [S1] emphasized the dam size based on the client's request, and they also analyzed the total approved budget. The budget was used to guide the criteria of the dam, so here they are rethinking the criteria request by the client (chief of the village). [S1] agreed with the situation offered by his peer, and she interpreted it by clarifying the size and the budget.

Develop Decision

The "Decision" step is the final step in the EDP. The students worked in groups, comparing their design to the other group's design. In this step, the majority of CTS were self-regulated (see Table 4). Students presented the results of their final design, at the front of the classroom, to show that their design could effectively solve the problem. Furthermore, the students compared their designs, drawing a conclusion to redesign when their design failed. Table 4 shows Group 1's design compared to that of Group 2.

Table 4 Results of Students' Self-examination
Full size table

Students used this data to develop an improved design. Moreover, in this step, students also implemented a redesign to solve the problem given. This situation expressed high CTS in self-regulation criteria; student [A1] stated the advantages of their group's dam design.

[A1]: The dam made by our group uses the concepts of Pascal's Law and hydrostatic pressure, so that the construction can be durable and sturdy. It also has another advantage, namely the budget is not more than $2000. However, it still has a drawback, which is to look at the situation and condition of the cost of the dam.

Discussion

This study showed that EDP is beneficial in supporting students' CTS. Each stage of the EDP could be investigated, focusing on the majority of CTS exhibited. This result is in line with the results uncovered by Yu et al. (2020): the EDP plays an important role in developing CTS. When students work through all the stages of the EDP, they also develop and meet the criteria of CTS. Especially, when students did individual work, they described the cognitive domain based on the CTS; students identified the problem in the situation, highlighted the goal of the human need, and paid attention in the constraint to solve a problem (Ernst & Monroe, 2004). When students worked in the groups to discuss their ideas, they tried to exchange their ideas in the stage of planning a solution, tried their design, tested, and decided on their design (Giri & Paily, 2020; Kabir, 2002). This situation described the affective domain because students communicated based on their argumentation to reinforce their ideas showed an affective domain in CTS (Antonieta et al., 2005).

The goal in the EDP classroom was for students to make decisions regarding effective solutions to solve the lack of water in rice fields. Following the steps of the EDP, going from "Plan" to "Test," students clearly utilized the process of argumentation to decide. This implementation of the EDP followed reflective thinking because students also conducted self-regulation to redesign their solution. Self-regulation involves the necessity of re-establishing the performance that students think is recursive to setting the goal (Ghanizadeh, 2017). Furthermore, students thought back and forth between the problem given and the solution produced, following the learning cycle (Dewey, 1993; Garrison et al., 2001). In addition, during these activities, students designed a solution based on their understanding of the physics concepts learned.

This process highlighted that student can improve their CTS. Student [U1] showed that in the individual activities he had low CTS, but after joining the group discussion he had a high level of CTS. This phenomenon shows that group interaction can improve a student's level of CTS through communication (Farmer & Wilkinson, 2018). Students' communication regarding the design of the solution showed the concept of argumentation because students provided evidence to support their claims (Mathis et al., 2017). Therefore, students argued using a well-founded reason from various sources, including discussion activities, which in turn require CTS (Yazici et al., 2020).

This study emphasizes that in the EDP stages could express the specific criteria of student's CTS. Additionally, the contribution of this study is that it seems essential to show that measuring of CTS need student’s performance to conform achieving in CTS. Mainly, the cyclical thinking and the use of self-regulation could improve solutions based on the given problems. Using the EDP also gives students an opportunity to communicate with each other to build a better solution based on physics concepts. Furthermore, this research also asked students to rethink the management strategies for solving problems via communication in the group that described CTS based on the affective domain. This research differed from previous research that has investigated the link between the EDP and students' CTS by giving students post-tests of CTS (Mutakinati et al., 2018; Yu et al., 2020). Students could be investigated in more detail using cyclical thinking to generate the best solution based on the problem given and the constraints (Arık & Topçu, 2020; Lottero-perdue et al., 2015; Tank et al., 2018).

Conclusion

This research provides, through a qualitative study, empirical evidence that the EDP supports a student's CTS. In the case study, students utilize their CTS based on the dominant criteria that appeared in each step of the EDP. The EDP facilitated students' collaboration, working in groups, where students could share and explore their ideas. Additionally, students engaged in argumentation while they began the phases of planning, trying, and testing. After students decided on their design to solve the problem, they conducted self-examination, looking over their design, seeing if the results were comparable to other groups. This situation demonstrated iterative thinking, which is one of the goals of effective CTS.

This research described the infusion of engineering as central to integrating science, technology, engineering, and mathematics (STEM) approaches. This study implies that CTS should be measured during the learning process. For the teacher, the EDP approach might be formulated to teach integrated STEM in the future. Through EDP stages, the integration subjects could be involved in the classroom to support the students' 21st-century skills. The policymaker needs to support the implementation of EDP in the school curricula because students showed positive behavior in the CTS. Additionally, providing professional development (PD) in implementing engineering education is also essential to running the EDP in the classroom. Through this PD, the application of EDP in STEM learning will improve both in quality and quantity.

Furthermore, this study was limited in the sampling of participants, but the exploration of CTS in EDP stages could be described in detail. Analysis in the greater participants needs to be assessed to show the results consistently. Moreover, the EDP is to infuse the engineering in STEM education, so that in the future research, the analysis of student’ CTS through STEM learning can be investigated to gather student’ understanding comprehensively in science, technology, engineering, and mathematics subjects than in silo subject.