Concept Mapping as a Metacognition Tool in a Problem-Solving-Based BME Course During In-Person and Online Instruction

Abstract

Metacognitive skills can have enormous benefits for students within engineering courses. Unfortunately, these metacognitive skills tend to fall outside the content area of most courses, and consequently, they can often be neglected in instruction. In this context, previous research on concept mapping as a teaching strategy points to meaningful learning. The purpose of this innovation paper is to report an application of concept mapping (1) to facilitate metacognition steps in students, and (2) to identify the muddiest points students struggle with, during both in-person and online instruction of a problem-solving-based biomedical engineering course. This innovation article also looks at the usefulness of concept mapping through instructor and student perceptions and students’ class performance. The entire concept mapping intervention was conducted during weeks 8–10 of the Spring 2019 in-person quarter and during weeks 3–4 and 8–10 of the Spring 2021 online quarter. The exercise involved concept mapping, explanation and discussion with peers, and answering structured reflection prompts. Each concept map activity was contextualized to the metacognitive knowledge domain of the revised Bloom’s taxonomy. The average class performance was compared between students who completed concept mapping vs. those who did not, using a t-test and one-way ANOVA at alpha = 0.05 significance level followed by a Tukey HSD test. Students’ concept maps and reported answers were analyzed qualitatively following the concept mapping intervention. During the Spring 2019 in-person quarter, 59.30% of students completed concept mapping with reflection, whereas 47.67% completed it in spring 2021 online instruction. A two-tailed, unpaired t-test indicated that concept mapping did not significantly enhance students’ class performance (p > 0.05) within each of the in-person and online instructions. Peers’ suggestions to students to improve concept maps revealed themes related to course concepts, prerequisite concepts, and the act of concept mapping itself. Concept mapping was effective in revealing the muddiest points of the course. Concept mapping did not significantly enhance students’ class performance either in-person or online instruction (effect sizes were 0.29 for the 2019 in-person quarter and 0.33 for the 2021 online quarter). However, instructors and students’ perceptions reflected that concept mapping facilitated metacognition in a problem-solving-based biomedical engineering course both during in-person and online instruction. Most students (78%) were optimistic about the usefulness of concept mapping for this course, and 84% were inclined to apply it for a variety of other courses.

Introduction

Background

One of the major intellectual challenges students face upon entering college is managing their learning.⁵⁶ Students who perform well in high school can often not show the same performance level in the university. They might struggle to adapt to a new learning environment and may not know strategies to cope with more complex tasks. They tend to assume that they know how to study based on their high school experiences (memorize facts rather than understanding key concepts and ideas) without realizing that they may need to adopt a different approach as the learning outcomes may have changed.²⁷ To worsen the problem, students may not realize their approach could be at fault.⁴³ Consequently, they may relentlessly redouble their previous efforts after a poor performance on a midterm only to find in the final examination that the fundamental problem lies within their strategy, and more of the same surface learning strategy does not help.

At a large public university, while teaching a problem-solving-based course on mass and energy conservation principles, called “Fundamentals of Bioengineering” to sophomore biomedical engineers, we noticed that many students were taking an approach of memorizing equations and reciting problem-solving steps the instructor used for specific examples in lectures, rather than concentrating on the conceptual underpinnings. This course serves as a prerequisite course for the upper-division courses in biomedical engineering in which subsequent courses rely on students’ content knowledge from this course. Therefore, we are continually improving methods to make content more accessible from a learning perspective and enhance the conceptual understanding of students.

We wanted our students to adopt deeper learning approaches. We discovered in the literature that of the three common approaches to learning—surface, deep, and achieving—the last two result from effective metacognition.⁷ Metacognition, the process of reflecting on and directing one’s thinking,⁴⁵ can help students become self-directed learners, breaking the cycle of relentless efforts made in a direction without assessing what one is doing wrong. Specifically, metacognition can help in self-directed learning by teaching one to assess the demands of a task, evaluate one’s knowledge and skills, plan an approach to tackle the tasks based on the knowledge and skill assessment, monitor the progress as a result of using a particular approach, and adjust the strategies as needed.¹³

However, these metacognitive skills do not necessarily develop independently in students. Instructors play an essential role in helping students cultivate these skills and help students become expert learners who use the knowledge they have gained of themselves as learners, of task requirements, and specific strategy-use to deliberately select, control, and monitor learning to achieve desired goals.¹⁸ Unfortunately, the assumption that metacognition may take ample curriculum space, time, and effort can result in anxiety leading to the omission of metacognition in the instruction. Consequently, many students lose on cultivating appropriate intellectual habits valuable across disciplines (such as planning one’s approach to a large project, considering alternatives, and evaluating one’s perspective). They do not unlock their full potential to gain in-depth knowledge of a discipline/topic covered within the curriculum.

To develop meaningful learning amongst our students, we were inspired to develop a strategic concept mapping intervention facilitating metacognition in students. We also leveraged the benefit of concept mapping in formative assessment, where students revealed the muddiest points of the course. In this context, concept mapping could support two features for quality teaching⁵⁰: (1) identifying the learner's specific existing conceptual and propositional knowledge, and (2) planning appropriate organization of new knowledge to optimize the learner's ability to relate to previously understood concepts and propositions.

We also hoped that concept mapping-based intervention would ultimately help students refine their problem-solving strategies. Problem-solving is a complex activity involving various components that include concepts, rules, and principles. However, it also involves structural knowledge, ampliative skills, and metacognition abilities. From this aspect, concept mapping was also considered a valuable tool for enhancing metacognition and problem-solving abilities.^34,35,65

Prior research related to the use of concept mapping for metacognition is overviewed in the literature review section. Based on this research and reported benefits, we applied the concept mapping intervention to both in-person and online instructional settings and tailored it to facilitate metacognition and revelation of the muddiest points in our problem-solving course.

The overall learning outcomes associated with concept mapping exercises in our course were to enable students to:

(1) organize the concepts learned in the course in a systematic way, (2) describe the connections between various concepts learned in the course, (3) create a visual map depicting the relationships between principal concepts, (4) verbalize their understanding by explaining concepts to peers, and (5) evaluate one’s strengths and weaknesses in explanation of the concepts.

Specifically, for metacognition, we wanted our students to be able to (6) assess the demands of a given task, (7) evaluate one’s knowledge and skills, (8) create a plan/an approach to tackle the tasks based on the knowledge and skill assessment. Nevertheless, an initial understanding of how students approach concept maps will be instructional for improving implementation of the activity for future courses, and inform setup of future studies.

We launched the first concept mapping intervention in BIM 20 class during the Spring 2019 in-person instruction (pre-COVID-19 pandemic). It helped the instructional team to know whether students were developing a coherent and scientific understanding of the important concepts before conducting the examinations. In 2020, due to the COVID-19 pandemic, course instruction was forced to be online. Therefore, we redesigned the concept mapping activity to suit an online class environment and relaunched it during the Spring 2021 online instruction. We share the results from both in-person and online implementation of concept mapping activity in our classroom in this innovation article. The context of different course implementations could indicate the robust nature of concept mapping for facilitating personal metacognition.

Literature Review

Metacognition

Metacognition is often referred to as “thinking about thinking,” which involves the regulation of one’s cognitive activities in the learning processes.²⁰ More specifically, metacognition consists of two dimensions, our knowledge, and regulation of our thinking processes.¹³ Knowledge of cognition comprises knowing about persons, tasks, and strategies. Regulation of cognition puts our knowledge into action through planning, monitoring, controlling, and evaluating activities.¹³ There is cyclical interaction of knowledge and regulation of our thinking processes in metacognition.

Metacognition is associated with several terms, such as metacognitive beliefs, metacognitive awareness, metacognitive experiences, metacognitive knowledge, feeling of knowing, a judgment of learning, theory of mind, metamemory, metacognitive skills, executive skills, higher-order skills, metacomponents, comprehension monitoring, learning strategies, heuristic strategies, and self-regulation.⁶⁸ While there is a consistent acknowledgment of the importance of metacognition through these terms, there have been inconsistencies in its conceptualization within learning.⁶⁸ A more unified model was given by Nelson,⁴⁶ who postulated that flows of information derive from two levels in metacognition. The first is an object-level, at which the cognitive activity occurs, and the second is a meta-level, which governs the object-level. Information about the state of the object-level is conveyed to the meta-level through monitoring processes, while instructions from the meta-level are transmitted to the object-level through control processes. Thus, if errors occur on the object level, monitoring processes will notify the meta-level, and control processes will be activated to resolve the problem. This simple and elegant model applies to metamemory and the phenomena of Feeling-of-Knowing and Judgment-of-Learning.^15,44 Other scholars have investigated Feeling-of-Knowing and Judgment-of-Learning as metacognitive processes.¹⁶

While everyone is metacognitively active to one degree, we can benefit from improving our metacognitive skills. Metacognitive instruction appears to enhance metacognition and learning in a broad range of students, but it is particularly relevant to students with insufficient studying practices.⁶⁸ Many students mainly use surface approaches to learning, such as rote learning, rehearsal, and memorization. These students would greatly benefit from a more elaborative and organizational approach associated with deeper and meaningful learning facilitated through tools helping in metacognition.

Concept Mapping to Enhance Metacognition and Problem Solving

One such tool facilitating metacognition is concept mapping, largely credited to Joseph Novak.⁵³ Novak et al. developed a concept mapping strategy at Cornell University in 1984.⁵³ Concept maps are a graphic organization technique designed to help learners explore their knowledge or understanding of topics that are highly elusive and mystifying.⁵³ Essentially, concept maps are diagrams that present the mental connections and association of structures of student knowledge. In its simplest form, a concept map connects only two concepts by a linking word.⁵³ A concept map consists of nodes representing concepts and links to show relationships among concepts. These nodes and links are arranged in a structure (hierarchical, cyclic, or hybrid) to represent all the key concepts. This graphical nature of the concept map can be useful to activate prior knowledge, support problem solving, enhance conceptual understanding, and organize and revise knowledge.⁷⁰ Early uses of concept mapping were mainly in the context of science classrooms. More recent uses have broadened to explore the nature of learning in various disciplines and contexts,²⁸ as well as for research productivity.⁴¹

Employed as a learning activity, concept mapping is claimed to be effective in two ways: as a cognitive strategy to stimulate learners to make cognitive progress in organizing and understanding new information⁵¹; and as a metacognitive strategy to empower learners to monitor and control cognitive progress.^32,66 The theoretical power of concept mapping is derived from Ausubel’s⁵ idea of meaningful learning. Ausubel’s learning theory places central emphasis on the influence of students’ prior knowledge on subsequent meaningful learning. Ausubel⁵ postulated that meaningful learning is a process in which new information is related to an existing relevant aspect of an individual’s cognitive structure. Novak demonstrates how concept maps put into practice the theoretical principles of Ausubel’s assimilation theory.⁴⁸ Novak describes how, during the evolution of a concept map, learners continually develop new propositions to elaborate and refine concepts that the learners already know. The new and more specialized concept is then subsumed into more general concepts in the learner’s cognitive structure. Thus, with new learning, a concept map helps elucidate how learner’s concepts are modified and integrated into a progressively more complex conceptual framework.

By explicitly identifying concepts and the relationships between concepts, concept mapping not only facilitates the development of a learner’s representation of domain knowledge but also facilitates the reflection on this progress. To establish a nonarbitrary association between the new information and the relevant concepts or propositions they already possess, learners are required to engage in an analytical process in which they evaluate, integrate, and elaborate on their understanding in new ways during the construction of concept maps. In this reflective process, concept mapping becomes a way “to learn how to learn,” as described by Joseph Novak et al.⁵³; it serves as a metacognitive tool to help learners take charge of their meaning-making.

Many researchers have investigated the effectiveness of concept mapping as a metacognitive tool.^{11,21,31,37,49,58,65} Concept mapping has been shown to effectuate self-reflection and strategic action by students in college,⁴ high school,¹² and primary school.⁶⁴ Drawing a concept map requires students to engage in higher cognitive functions.^1,67 Researchers report various benefits that this approach yields, including classroom shifts in the epistemological authority from the teacher to the student, less emphasis on right and wrong answers, creation of visual entry points for learners of varying abilities, and reduction of cognitive load to support learning.⁵⁸ August-Brady⁴ showed that students who used concept mapping demonstrated an increase in deep approach to learning and the self-regulation of that learning, compared with students who did not use concept mapping. Chularut and DeBecker report statistically significant gains in students’ level of English proficiency when concept mapping was employed as a learning strategy with students taking English as a Second Language.¹²

From the problem-solving perspective, compared to traditional teaching methods, concept mapping strategy has been found to significantly improve students’ performance in problem-solving.^9,36 Concept mapping provides a way to externalize students’ mental models.⁶⁵ Kamble et al. found that most of the third-year undergraduate mechanical engineering students were optimistic about using concept mapping strategy in an internal combustion engine course.³⁶ Their students indicated that concept maps could help them understand, identify, and connect the various concepts. Similarly, Stoymov⁶⁵ reported that concept mapping for students’ problem-solving specifically added value in idea generation and selection phases for solving ill-structured problems. Furthermore, although Zwaal and Ottingdid did not find empirical evidence, their report indicated that concept mapping might be a valuable tool to enhance the process of problem-based learning.⁷⁰ In their study, students working with concept mapping were more satisfied with the decision-making process and the communication within the group while solving problems, than the students who did not use concept mapping.

As a tool to help learners organize their cognitive frameworks into more integrated patterns, concept mapping also has its theoretical strength in meaningful learning to empower learners to transfer learning (apply something learned to other situations). The metacognitive view holds that successful transfer occurs when the problem solver can recognize the requirements of the new problem, select previously learned specific and general skills that apply to the new problem, and monitor their application in solving the new problem.

Despite these promising results of concept mapping as a tool for improving meaningful learning, there has been some pushback to concept mapping from students or their teachers.^6,54 Researchers have found that the actual value of concept mapping as a learning strategy depends on it being utilized appropriately by the learners. Hence, the setting of the classroom climate for the implementation of concept mapping is crucial. Indeed, existing literature suggests that the acceptance of the concept map strategy by both teachers and students depends on the appropriate time for introduction and identification of apt conditions for student utilization of the strategy.^38,39,55,61 We built such a classroom environment and carefully developed the concept mapping intervention to serve as a metacognition tool for students in both face-to-face and online course settings.

Concept Mapping for Classroom Assessment

Concept mapping enriches the practice of teaching by offering to meet the diverse learning needs of students, assess different types of knowledge (structural knowledge in addition to declarative and procedural knowledge), and implement different forms of assessment,³ e.g., diagnostic, formative and summative assessment. At the beginning of instruction, concept maps allow the teacher to identify what knowledge students already possess (diagnostic assessment) and to establish a baseline for future formative assessment activities. During instruction, concept maps can be used to evaluate changes in students’ structural knowledge and alter the instruction accordingly (formative assessment). At the end of instruction, concept maps can be useful to evaluate the final knowledge structure acquired by the students (summative assessment) and can tell the effectiveness of formative feedback activities.

Concept mapping is a suitable tool for formative assessment of students’ structural knowledge because concept maps represent the differences between the structural knowledge of experts and novices, supporting the characteristics mentioned in the definition of formative assessment. For e.g., concept maps allow differences to be seen between experts’ and novices’ structural knowledge in quantitative (e.g. number of linkages and concepts) and qualitative (e.g. the quality of linking phrases and uncovered relationships) aspects of concept maps.³

This broad feedback on the qualitative aspects of students’ structural knowledge can provide information about any misconceptions, false beliefs, and gaps in knowledge. Students can use this information to improve the state of knowledge, the level of achievement, and their learning process. In the case of the teacher, the feedback gives valuable information about the knowledge state of both specific students and the student group. This feedback can be used to make adjustments to the course.

Innovation Aspect

Previous studies have shown that concept mapping as a metacognitive tool plays a key role in students’ development of meaningful learning.^26,31,47,61 Our report provides new context to the body of knowledge by applying concept mapping as a metacognitive tool in a problem-solving biomedical engineering course in both a physical (face-to-face) and an online setting. We further report how employing the concept mapping intervention enabled formative classroom assessment, and how it enabled capturing students’ thinking about the utility of concept mapping.

Assuming that concept mapping can have real value for promoting meaningful learning and promoting metacognition under appropriate classroom ecology, we developed a specific class environment conducive to the use of concept mapping in our problem-solving-based sophomore BME course on mass and energy conservation principles. We then introduced the peer discussion component of the concept map exercise to channel students’ thinking about complex course concepts, grow and modify their concepts, and reflect upon their course knowledge organization. Concept mapping intervention in this report refers to the full intervention of creating a map + peer discussion + answering reflection questions. We also reframed the activity to suit both in-person and online classroom instruction.

To facilitate meaningful learning for our students, we represented each of the concept mapping activity steps to the cognitive process dimensions in the revised Bloom’s taxonomy.^2,40 We then investigated the effect of applying concept mapping on students’ class performance in both in-person and online instruction through their assignments and their self-reported perceptions. Finally, we utilized concept mapping activity for formative class assessment revealing student-perceived muddiest points in the course and took remedial action in response to the feedback.

Implementation

The broad learning outcomes for our “Fundamentals of Bioengineering” course are given below.

At the end of this course, students will be able to:

• 1. Methodically formulate a biomedical engineering problem and solve it

• 2. Develop mass and energy conservation equations

• 3. Apply conservation equations to model biological and physiological systems and to solve problems in biological and medical sciences

This four-credit course met twice a week for 80 min in the classroom and once a week for 50 min for discussion sessions (problem-solving sessions). This course encouraged pre-reading, active learning, and peer collaboration through various activities and assignments. For the Spring 2019 in-person instruction, the course assignments and grade weightage were—Reflection on assigned readings (5%), Quizzes based on the reading textbook (10%), Homework (10%), CAD project (15%), Midterm 1 (15%), Midterm 2 (20%), Cumulative Final Exam (25%). The concept mapping was offered as an extra-credit intervention. Concept mapping intervention was offered for extra credit to incentivize students to benefit from it without feeling extra stress since this was a new assignment introduced into the course. 59.30% of the class participated in the extra-credit concept mapping activity during in-person instruction in Spring 2019.

For the Spring 2021 online instruction, the course assignments with their weightage were—Weekly discussion posts (5%), Quizzes based on the reading textbook (10%), Homeworks (40%), Take-Home Midterm (20%), Take-Home Final Exam (25%). Concept mapping intervention was implemented as two concept mapping activities, the first one as a compulsory activity through the discussion post in week 4—teaching students what is entailed in a good concept mapping. 100% of the class participated in the first concept mapping intervention created as a required assignment in week 4 of the quarter. However, the second concept mapping was offered as an extra credit activity in week 10 to incentivize students to benefit from this metacognition tool for their upcoming final examination performance rather than be stressed about it. 52.33% of the class took advantage of this extra credit opportunity.

The overall timeline indicating where we embedded various concept mapping activities during the Spring 2019 in-person instruction and the Spring 2021 online instruction is shown in Fig. 1.

Figure 1

The timeline indicating the progress of concept mapping activities is indicated by the upper arrow in (a) for Spring 2019 in-person instruction, and the lower arrow in (b) for Spring 2021online instruction.

During the Spring 2021 online quarter, we thought offering an early introduction and extra practice of concept mapping in the course would benefit the students to construct their final concept map in week 9 in a more efficient manner and aid in metacognition.⁶⁸ To familiarize students with concept mapping early on in the course, we chose week 3 to facilitate an online TA-led concept mapping discussion on mass conservation principles for the first concept map. We also illustrated what constitutes a good concept map vs. a bad concept map.⁵² Students could use these concept maps for their take-home Midterm examination.

After providing students with time to cover the energy conservation principles in the syllabus, the second concept mapping intervention was done in week 9 of the 11-week quarter. Week 9 was selected in the hope that the concept mapping activity could be helpful to the students for their final examination in week 11.

Thus, apart from introducing concept mapping early in the course, the other major differences between the Spring 2021 online quarter and the previous in-person instruction were that the online instruction included an extra practice of concept mapping with peers (week 3); illustration of good concept mapping techniques during week 3, and more structured reflection with prompts designed to facilitate metacognition during the online Spring 2021 quarter.

The implementation of concept mapping in the in-person and online quarters is described in detail below. For results in which student comments are provided, pseudonyms are used to anonymize their responses.

Concept Mapping During In-Person (Face-to-Face) Instruction

During the Spring 2019 in-person instruction, students were asked to prepare a pictorial concept map in week 8, out of the 11-week quarter course, after completion of midterms. Students were instructed to develop the concept map based on at least 30 keywords related to mass and energy transfer concepts (Fig. 2) that they deemed important in chapters 1–4 of the course textbook, Bioengineering Fundamentals, authored by Dr. Ann Saterbak et al.⁶² Chapter 1 of this book introduces engineering calculations; chapter 2 deals with the foundations of the conservation principles; chapter 3 elaborates on the conservation of mass; and chapter 4 discusses energy conservation. We provided students with example words and illustrated the process of making a concept map in class using a handout (Supplementary Information), along with examples of good and bad concept maps.⁵² The concept mapping strategy was presented to the class as a graphical tool for organizing and representing course information and students’ ideas. The learning outcome for this class was: At the end of this class period, students will be able to create their concept map for mass conservation principles. Accordingly, students were asked to construct their concept maps and encouraged to be creative in representing the concepts with the help of equations/pictures. Students were also encouraged to draw concept maps as nodes and links in a network structure, where nodes represent concepts, usually enclosed in circles or boxes, and links represent relationships, usually indicated by lines drawn between two associate nodes. It was emphasized to write linking words or phrases specifying the relationship between the two concepts on the linking lines between them. Students began making their concept maps in class, and we allowed a week’s timeframe to finalize their maps. Students were asked to bring their concept maps to next week’s class for peer discussion.

Figure 2

Figure adapted from our work (anonymized).

The concept maps in the Spring 2019 in-person quarter provided a neat and unique graphical view of how students organized, connected, and synthesized their ideas about mass and energy conservation principles. Figure 2 shows two representative concept maps by two different students. Part (a) shows a representative concept map where the student made ample use of equations, part (b) shows a representative concept map where the student made ample use of cartoon drawings. However, the quality of linking words was reduced compared to the map type shown in (a).

The following week (week 9) of the instructional quarter, an entire class time (80 min) was devoted to concept map discussion. The learning outcome for this class was: At the end of this class period, students will be able to explain their concept map for mass and energy conservation principles and reflect on their knowledge gaps. Eighty-six students of the online class were divided into 14 groups (about six members in each group). Each team was asked to assign a scribe and a group reporter. The instructor and the teaching assistant (TA) provided teams with easel pad stations, markers, post-it notes, and instruction papers. Fourteen easel pad papers with sticky sides were pasted on the walls of the classroom.

For group discussion, each student in the group was asked to describe their concept-map to their groupmates (5 min for each student between six students for a total of 30 min), and this was followed by feedback from their peers about which concepts were not explained clearly by the presenter (3 min for each presentation from six students for a total of 18 min). The group reporter took notes on the easel pad noting the feedback shared by peers for every presenter in the group. Then, for each difficult concept noted on the easel pad, all group members brainstormed alternative pictures/equations to help their peers explain the concept (10 min). The scribe helped illustrate the brainstormed points on the easel pad. At the end of the activity, the group reporter shared with the instructor their progress (5 min). The class’s remaining 10–20 min were reserved for them to explore other groups’ maps.

Following this class exercise, to facilitate metacognition, two prompts were provided to students to complete individually for extra credit, worth 0.1% of their final grade. The prompts were:

After this class exercise, please circle the concepts on your concept map that you struggled to explain to your group. Then, on the backside of your concept map, please write down answers to:

• (a) what about that concept was difficult to explain to peers?

• (b) what ideas did you get from the group to understand this concept better?

Of 86 students, 51 students responded to the prompts and turned in their concept maps for extra credit. From prompt responses, the keywords representing student-reported difficulties for explaining to peers were shortlisted for every response. The frequency of the concepts mentioned as difficult to explain was accounted for by generating a word cloud. Student responses from prompt (b) were analyzed to determine how the discussion with peers might have helped them approach difficult concepts better. The concept map study was approved by IRB (Protocol: 1730137-1 for Spring 2019).

Concept Mapping During Online Instruction

During the Spring 2020 quarter, because of the COVID-19 pandemic, the course met the significant challenge of converting to an online teaching mode in a short period of a week instead of planned face-to-face instruction. The concept mapping exercise was not implemented that year in the hustle to convert instruction online.

However, for teaching the same online course in Spring 2021, we had a year to plan and integrate concept mapping into the online format, resulting in an advantage over the Spring 2020 quarter. The instructor applied the following method to integrate concept mapping in an online quarter.

Students were informed about the Concept Mapping exercise in week 1 of the quarter while going over the syllabus. In week 3 of the quarter, two teaching assistants (TAs) facilitated a class-wide concept mapping exercise using an instruction template.²⁹ The process of making a concept map was illustrated by the teaching assistants in class for about 30 min, using PowerPoint slides and google slides, along with examples of good and bad concept maps, leaving the rest of the class time for making concept maps by students.

For individual student concept mapping, after the TA-illustration of concept mapping, students in the online class were grouped (randomly) in ZOOM breakout rooms (3–4 students per room) and given 30 min to start drawing individual concept maps on their own. They could discuss the concepts on mass transfer principles that they had learned in the course or discuss the logistics of making the concept map in their groups. TAs acted as facilitators for this activity, rotating through various breakout rooms in 30 min of the concept mapping activity. During this time, students were encouraged to use nodes and links, where the nodes represented concepts, usually enclosed in circles or boxes, and links represented relationships, usually indicated by lines drawn between two associate nodes. Words on the line are referred to as linking words or phrases specifying the relationship between the two concepts. Students were given a week to revise the concept map. After a week, they were individually asked to post their finished concept map to Canvas Learning Management System (LMS) as a response to a Discussion Assignment with the prompt: “What is the muddiest point in your concept map and what can be done to clarify it?”.

All 86 students completed the discussion post. The research team looked at the muddiest point data to create a word cloud of concepts marked as the most difficult ones to understand by students.

Further, in weeks 9 and 10 of the instructional quarter, students were given the following assignment involving peer discussion and self-reflection components for 1% extra credit. Out of 86 students, 45 students turned in this assignment. The structure of this assignment is given below.

Extra Credit Assignment

Part 1: Implemented in Week 9

• Design a new concept map on Ch 3.7-3.9 and Ch 4 concepts, using the technique you learned in lecture 6.

• On Thursday 5/27, after the first half of the class, you will be explaining your concept maps to your peers, and listening to theirs, in break out-groups.

• You will give feedback to your peers about what terms in their explanation were not clear. Similarly, you will get feedback for your explanation.

• After the breakout groups, please circle/highlight/ underline the concepts that you were struggling to explain.

On a separate paper, for each concept you struggled to explain in your group, write a paragraph answering each of the following prompts as Extra Credit—part 2.

Part 2: Implemented in Week 10

• (a) What about that concept circled in your map was difficult to explain to your peers in the breakout group?

• (b) What ideas did you get from the group to explain that concept better?

Now, please answer the following open-ended questions:

• (c) Has concept mapping in this course so far been useful to you? How (in what ways) did it prove to be useful?

• (d) Would you be using concept maps again for any other courses? Any examples you can think of?

• (e) Would you recommend concept mapping to your peers in other classes? Why or why not?

• (f) If anonymized completely, do you consent to use your concept map and associated answers to this assignment as a representative example to help demonstrate the use of concept map in learning to other educators /learners within OR outside of the [university]? (Yes / No)*

*Your choice to decline will not affect your grade or performance in this course in any manner.

The prompts a-e were designed to facilitate metacognition as described in Table 1. Each concept mapping activity step was mapped to the cognitive process dimension in the revised Bloom’s taxonomy.^2,40 The prompts (b–e) focused on making students self-reflect on their concept maps more deeply. These prompts were constructed to enable students to (a) identify the concept(s) that they found difficult to map /explain to peers in breakout groups; (b) reflect on the ideas suggested by peers for better grasping and explanation of the identified difficult concept(s); (c) reflect on the usefulness of the concept mapping activity for this course; (d) contemplate on the concept mapping application to other courses; (e) consider the recommendation of concept mapping activity to peers in other classes.

Table 1 Mapping concept mapping intervention to metacognitive knowledge in cognitive process dimension of revised Bloom's taxonomy.

Thematic Coding of Students Responses in Online Instruction

From prompts (a) and (b), the difficult concepts students reported in explaining to peers were thematically coded.²³ We started with a set of 41 student responses to prompts (a) and (b) and listed keywords highlighting difficulties students reported in explaining concept maps to peers and suggestions they got from peers for improvement. We further analyzed these keywords concerning where and when the muddiest concept reported was first introduced to students. Specifically, from responses to prompt a) “What about that concept circled in your map was difficult to explain to your peers in the breakout group?”, three themes emerged—(1) difficulties reported in topics that were specific to this course, (2) difficulties that pertained to topics taught in prerequisite courses, (3) difficulties that related to other topics (non-prerequisites). Here, prerequisite courses included a general chemistry series, calculus and vector analysis, thermodynamics, and an introduction to MATLAB. Non-prerequisite courses include courses students previously taken as part of the undergraduate curriculum but are not explicitly required for this course, such as linear algebra, organic chemistry, and differential equations. Notably, the coding determination was decided after students performed the exercise. Nevertheless, the coding in this context could potentially highlight concepts that might not be addressed under the assumption that students understand prerequisites sufficiently.

Students’ answers to prompt (b) “What ideas did you get from the group to explain that concept better?” were also similarly thematically coded into three areas—(1) ideas based on interpretation of topics specific to this course, (2) ideas based on topics students learned in previous courses, (3) ideas students learned in class about making a good concept map.

Additionally, a word cloud was generated based on the frequency of the concepts mentioned as difficult to explain in part a. The prompts (b)–(e) responses were also analyzed qualitatively, and representative student responses were reported. The Spring 2021 concept map study was approved by IRB (Protocol 1777033-1).

Comparison of Student Performance During In-person and Online Instruction

To test whether there was an impact of concept mapping done during in-person vs. online instruction on students’ class performance, we performed a one-way analysis of variance (ANOVA) on class performance means. Specifically, comparisons were made for students who completed concept mapping and those who did not participate during the Spring 2019 in-person instruction and Spring 2021 online instruction. We performed Tukey HSD/Tukey-Kramer posthoc test to determine if groups were significantly different.

Results and Discussion

Flavell¹⁹ defines metacognition as “one's knowledge concerning one's own cognitive process and products or anything related to them." The definition classifies two aspects of metacognition: knowledge about cognition and regulation of cognition. Knowledge about cognition concerns knowledge about one's cognitive resources, and regulation of cognition concerns self-regulatory mechanisms used by an active learner during ongoing attempts to solve problems. Concept mapping is claimed to engage a high degree of metacognitive involvement^32,49 because the learners are aware of and are active in their knowledge construction. Our course focused on promoting metacognitive knowledge by mapping the steps in concept mapping intervention to cognitive process dimensions of the revised Bloom's taxonomy. The regulation of cognition and self-regulatory mechanisms students use after concept mapping intervention were not characterized for this report.

Promoting Metacognitive Knowledge via Concept Mapping

The revised Bloom taxonomy^2,40 consists of two dimensions: (1) knowledge (or the kind of knowledge to be learned) and (2) cognitive process (or the cognitive processes to be used in acquiring knowledge). Based on Anderson's perspective, the knowledge dimension is composed of four categories: Factual, Conceptual, Procedural, and Metacognitive. The second dimension is the cognitive process, involving six levels: Remember, Understand, Apply, Analyze, Evaluate, and Create.² These six categories are arranged in a hierarchical framework, building in the complexity of the skills.

The metacognitive knowledge in Krathwohl’s framework involves knowledge about cognition in general, as well as awareness of, and knowledge about one’s own cognition.⁵⁷ It means consciously thinking about oneself's understanding to know about cognition and to also know how to regulate one’s cognition.⁶⁹ Metacognition in a problem-solving course can thus be strategic or reflective knowledge about how to go about solving problems, cognitive tasks, including contextual and conditional knowledge and knowledge of self.⁶⁹

A concept map has the potential to address both the metacognitive knowledge dimension in Bloom’s taxonomy as well as the cognitive process dimension. Cognitive processes are addressed as a student has to work hard to complete a concept mapping task, including structuring of knowledge, re-constructing/re-combining it to answer such questions as which concepts are interrelated, realizing what kinds of relationships exist between concepts, and which terms/keywords display knowledge mastered in the study course. Moreover, concept maps improve students’ mental capacity to understand the information, to process it in a meaningful way, to retain it, and to retrieve it from memory when the information is needed. Therefore, concept mapping activates and facilitates students’ cognitive processes as well as allows them to develop meta-cognitive reasoning skills: for example, if the student cannot relate some concepts to other available concepts, the student can ask him/herself what s/he does not know.

Focusing on these metacognitive aspects, we specifically mapped various steps of our concept mapping activity to cognitive dimensions in Bloom’s taxonomy as illustrated in Table 1. Each step as it relates to various hierarchical level(s) in the cognitive process dimension in revised Bloom's taxonomy is described in the Supplementary Information.

Concept Mapping During Spring 2019 In-Person Instruction

Of 86 students, 51 students (59.3% of the class) responded to the prompts and turned in their concept maps for extra credit in week 9. As students were introduced to new concepts in our course, they embarked upon a cognitive process to construct meaning and make sense of the new ideas and consciously or subconsciously integrate these new ideas with their existing knowledge. When students drew concept maps, they gained insight into what they already knew and how they could represent their knowledge, working through all six levels of revised Bloom’s taxonomy.⁴⁰ Novak and Gowin⁵³ as well as other scholars^11,25,37 have emphasized that concept mapping requires students to work at all six cognitive levels of Bloom’s taxonomy: Remembering, Understanding, Applying, Analyzing, Evaluating, and Creating. This advantage is especially manifested when students have to show linkages or cross-link various concepts in the concept map because when students begin to focus on them, they can see that each concept could be related to any other concept.

Apart from application to cognitive processes in revised Bloom’s taxonomy, another feature of our concept mapping activity was its low directedness, where students were free to decide which concepts should they include in their maps, which concepts are related, and which words to use to explain a relationship.⁵⁹ In contrast, high-directed concept map tasks provide students with the concepts, connecting lines, linking phrases, and the map structure. We chose low-directedness in our activity because scholars report low-directed techniques are content-rich (strong conceptual knowledge demand) and process open (student determines procedure), whereas high-directed techniques are content lean and process constrained.²⁴ Ruiz-Primo et al. demonstrated in their study how the low-directed technique provides students with more opportunities to reveal their conceptual understanding (explanations and errors) than the high-directed techniques.⁶⁰ We thought a similar low directedness of concept mapping aided in promoting metacognition in our students.

Concept Mapping During Online Instruction

Concept Mapping Activity for Mass Conservation Principles in Week 3

For the first concept mapping activity in week 3 of the Spring 2021 quarter, all 86 students submitted a concept map as an obligatory assignment on mass conservation concepts. About 10% of students created quality maps with detailed connections and thoughtful linking words written between concepts. Others either showed detailed connections with partially complete linking words or showed some clustering of ideas without linking words. All students were given full points for completing the concept maps without deducting points for partial linking or missing linking words, as the instructor focused on having students reflect on the concept map and identify the muddiest points. However, the instructor provided feedback in class on how to improve concept mapping in their future submissions.

The muddiest point analysis of students' concept mapping activity for week 3 is presented in word cloud format (Fig. 3). From student responses, most students identified difficulty in (1) determining when to use algebraic, differential, or integral forms of an accounting equation for a problem; (2) defining a system based on the problem description. An accounting equation was new to many students early in the course. Students were also having trouble interpreting the problem statement, which led to the emergence of the above muddiest points in their concept maps. As a remedial action after looking at the reported muddiest points, the instructional team selected specific examples to work out in class, discussion sections, and online quizzes. These activities aimed to help students get more practice with correct interpretations of problems by identifying systems and boundaries and applying the correct form of accounting statements.

Figure 3

Word cloud representing muddiest concepts reported by students in week 3 of the Spring 2021 online instruction. The size of a word shows how important it is, e.g., how often it appears in a text — its frequency. This word cloud was generated using https://worditout.com/.

Concept Mapping Activity for Energy Conservation Principles in Week 9

The second concept mapping activity was similar to the Spring 2019 quarter, except that the students were given additional prompts for self-reflection, and the syllabus did not cover the dynamic system portion for energy conservation problems. Like in-person instruction, it was also possible during online instruction to apply concept mapping to various cognitive processes in revised Bloom’s taxonomy, and preserve the low directedness,⁵⁹ where students were free to decide which as well as how many concepts should they include in their maps, which concepts to link, and which words to use to explain a relationship.

Representative work of students' concept maps is shown in Fig. 4. Overall, students' concept maps were more detailed compared to their week 3 concept maps. Students also showed richer connections of concepts with each other (e.g., Fig. 4a). Some students also wrote down formulae along with concepts in their maps and uniquely wrote examples about the concepts from the textbook (Fig. 4b). Few students continued to show a basic mapping structure with minimal nodes and connections to capture the concepts (Fig. 4c). However, there was an improvement in linking words on the map in this second attempt. Similarly, other scholars have reported improved final concept maps compared to initial maps, indicating an increase in students’ conceptual and critical thinking ability.¹⁴

Figure 4

Representative concept maps presented by students in week 9 of the course, focusing on the energy conservation principles. (a) concept maps with clear clustering of ideas, detailed connections, and linking words between clustered concepts, (b) hand-drawn concept maps with ample formulae/equations, (c) basic mapping structure with minimum branching, and minimal connections between the concepts.

Muddiest Point Analysis for In-Person and Online Instruction

The most common themes that students seemed to struggle with during the Spring 2019 in-person class are indicated in Fig. 5a. These themes included concepts of Enthalpy, Dynamic Systems, Energy Accumulation, Open systems with reactions, Heat of Reaction, use of Hypothetical path, and Limiting Reactants, among others. Many students knew the formulae for these concepts but did not understand in depth what the concept itself meant and how it was a tool in solving problems on energy conservation in various systems.

Figure 5

Word cloud representing concepts that students indicated as difficult to explain in their peer groups during (a) Spring 2019 in-person instruction and (b) Spring 2021 online instruction. The size of a word shows how important it is, e.g., how often it appears in a text—its frequency. This word cloud was generated using https://worditout.com/.

The visualization of the muddiest points reported by students in week 9 of the Spring 2021 online quarter is provided in Fig. 5b. Enthalpy again emerged as the most confusing concept, followed by the heat of reaction at non-STP conditions and the energy conservation equation. This year, dynamic systems did not emerge as a difficult topic because it was omitted from the syllabus for the Spring in 2021 quarter by the instructor.

Group discussions with peers seemed to have revealed to students some alternative ways of thinking about a concept rather than reciting it. Emphasis on these outcomes from the peer discussion and self-reflection of the concept mapping activity could trigger higher performance of students in the course. Nevertheless, this remains to be investigated through revised integration of this metacognition tool.

For instructors, it was highly beneficial to see themes in students' understanding development and knowledge around energy conservation concepts. It helped direct our teaching in Spring 2019 in-person and Spring 2021 online instruction. It was evident through students' concept maps that while some students created appropriate concept maps with rich content, elaborate connections, and linking words, some students had misclassified concepts of energy conservation by inappropriately linking unrelated concepts, failing to connect related concepts, or assigning an item to a superordinate position that belongs in a subordinate position. Thus, the concept map served as a powerful tool to ascertain what the learner already knew, including faulty knowledge structures or misconceptions, and helped organize the subject matter of new material to be taught. This use of concept maps to diagnose problems in students' knowledge organization has been reported by other scholars.¹

Class Performance Comparisons During In-Person and Online Instruction

The statistical comparison of class performance between students who completed concept mapping with reflection prompts vs. those who did not is indicated in Table 2. Specifically, for the Spring 2019 in-person instruction, an average student grade of 80.51% was observed for 51 students who completed concept mapping and reflection prompts vs. those who did not complete concept mapping with reflection prompts (average grade of 78.29% for 35 students in the class). However, a two-tailed, unpaired t-test for groups assuming unequal variance indicated no statistically significant difference (p > 0.05) between the means of students' final scores for students who either completed the concept mapping or did not complete concept mapping. The effect size for comparison of means between final scores of students who completed concept mapping vs. those who did not was observed to be 0.29 (Cohen's d).

Table 2 Comparison of means of final performance between students who completed concept mapping vs. those who did not.

A similar trend was observed for the Spring 2021 online quarter. Students who completed concept mapping and reflection prompts scored higher (average grade of 87.27% for 45 students) than those who did not complete concept mapping with reflection (83.10% average grade for 41 students). However, a two-tailed, unpaired t-test for groups assuming unequal variance again indicated no statistically significant difference (p > 0.05) between the means of two groups, i.e., the mean of students' final scores for groups who completed the concept mapping was not significantly different from those who did not complete concept mapping during Spring 2021 online quarter as well. The effect size for the Spring 2021 online quarter was 0.33 (Cohen's d).

Previous studies have reported that concept mapping used in conjunction with other educational strategies has led to superior achievements.⁴⁹ Although enhancement of class performance was not statistically established, a small effect size was observed along that direction. The effect size for comparison of means was slightly smaller for Spring 2019 in-person instruction (Cohen's d = 0.29) when one concept mapping activity was given compared to the Spring 2021 online quarter (Cohen's d = 0.33) when two concept mapping activities were conducted. It will be interesting to see if there is any correlation between the number of times students perform concept mapping with their performance in class in future work.

Although the unpaired t-test comparison of means did not indicate a significant difference between class performance means of students who completed concept mapping vs. those who did not, we also wanted to investigate how the online concept mapping activity impacted class performance compared to the in-person instruction. Therefore, we performed one-way ANOVA followed by Tukey HSD test on all four groups: (1) Group 1: Students who completed concept mapping + reflection during Spring 2019 in-person instruction; (2) Group 2: Students who did not complete concept mapping + reflection during Spring 2019 in-person instruction; (3) Group 3: Students who completed concept mapping + reflection during Spring 2021 online instruction; (4) Group 4: Students who did not complete concept mapping + reflection during Spring 2021 online instruction. One-way ANOVA followed by Tukey HSD test indicated that the difference between the means for Groups 1 and 3 and Groups 2 and 3 was significant at alpha = 0.05. This indicates that of the students who completed concept mapping with reflection (Group 1 and 3), online instruction (Group 3) demonstrated higher class performance than the in-person instruction (Group 1). Perhaps the extra practice the students received in concept mapping in weeks 3–4 of online instruction helped them enhance their class performance. This result was encouraging given the challenges of conducting online instruction during the pandemic. However, many other factors such as reduced syllabus, online classes, and diverse student populations could have contributed to this result.

We further looked at the content and structure of concept maps submitted by students to categorize them into poor vs. good/excellent levels of concept maps, as described by Canas and Novak.¹⁰ A concept map with good structural quality but poor content quality, or poor structural quality and good content quality was still considered poor. Accordingly, we found that 10 out of a total of 51 concept maps (19.6%) were at a poor level during Spring 2019 in-person instruction. This number of concept maps at a poor level surprisingly increased to 18 out of 45 (i.e. 40% poor) during Spring 2021 online instruction.

We next analyzed which quartile of class performance the authors of the poor-quality maps lay in for Spring 2019 in-person and Spring 2021 online instructions. Of the poor-level concept maps, most of their creators were within quartiles 1 and 2, indicating that students with lower-quality concept maps also scored low in overall class performance. However, a few students who created poor-level concept maps also scored within the upper quartiles (3 and 4) of class performance. We think this is because more knowledgeable students in class probably did not need a complex map; a simple map may have been sufficient to act as a set of keys to unlock their memory and reasoning store. This is in agreement with a study by Johnstone, which indicated that students who produce poor concept maps could fall into the lower and upper quartiles of normal assessment regimes.³³ The box plot summary indicating the quartiles and student performance means is shown in Fig. 6.

Figure 6

Box plot summary reflecting the distribution of students' overall class performance means for four groups: (1) Group 1: Students who completed concept mapping + reflection during Spring 2019 in-person instruction; (2) Group 2: Students who did not complete concept mapping + reflection during Spring 2019 in-person instruction; (3) Group 3: Students who completed concept mapping + reflection during Spring 2021 online instruction; 4) Group 4: Students who did not complete concept mapping + reflection during Spring 2021 online instruction. Q1 and Q3 denote lower and upper quartile levels. The numbers highlighted in yellow on box plots for Group 1 denote the number of poor level concept maps in each quartile of class performance during Spring 2019 in-person instruction. The numbers highlighted in green on box plots for Group 3 denote the number of poor level concept maps in each quartile of class performance during Spring 2021 online instruction. (Data shown in inset table).

Capturing Students' Thinking About Concept Mapping Activity in Spring 2019 In-Person Quarter

59.30% of the class students (51 out of 86 students) submitted responses to the provided prompts. From concept mapping activity results from the Spring 2019 in-person quarter, the instructor went through student responses to self-reflection prompts and aimed to understand how/in what ways students were internalizing the feedback received on their concept map explanations and reflecting upon their work. We looked at whether the peer presentation component seemed to help. The instructor found that peer discussion enabled students to identify and discuss similarities and differences in their maps and provided an opportunity to articulate their thoughts. As students verbalized their understanding and gave feedback to each other, they also clarified others' ideas. They gave each other a novel lens through which to look at the mass and energy conservation principles.

For example, one student who took in-person instruction in the Spring of 2019 wrote-My group suggested that I look back at lectures covering [section] 4.10, so that I have another resource to learn from, apart from the textbook. They also suggested that, after I read more about dynamic systems with energy accumulation and do some problem solving, I revisit this concept map and try relating dynamic state systems to more concepts (currently I only have the connection with the first law of thermodynamics drawn).

Furthermore, a student from the Spring 2019 in-person instruction wrote-The concepts that I circled in red were difficult to explain because I didn't have a solid understanding of them. For example, open systems with chemical reaction was troubling for me because I didn't understand the units of heat of reaction or how to use it. I forgot the formulas. Some ideas that I got from my peers was to relook at the formulas and write units down for each term and see how they cancel. That way I can check whether I wrote the units correctly and also I can remember it better. Overall this experience was good!

Another student commented-My peers asked me if I just memorize the equations, and I said yes. And they said that this is where the problem lies. Instead of memorizing equations, learn why an equation is like this, and don't plug and chug.

We also found that while peer discussion was important, it was equally essential to intervene and debrief the peer explanation exercise. Sometimes, the groups themselves did not have a clear idea of how to explain a concept better, and instructor intervention during and after the concept map discussion seemed to help students during the in-class discussion.

Capturing Students' Thinking About Concept Mapping Activity in Spring 2021 Online Quarter

Out of the total class of 86 students, 52.33% (45/86 students) completed concept mapping and submitted responses to the provided prompts in week 9. The thematic coding of student responses to metacognition prompts provided in week 9 of the online Spring 2021 quarter revealed that 78% of students found the concept mapping exercise to be useful because of the following reasons: concept mapping helped them compile concepts and make connections; it was found to be efficient for tests through summarization of concepts; it clarified ideas about topics and helps interconnect ideas; it helped decipher difficult concepts; it located knowledge gaps; it helped to focus, and refresh knowledge for tests; it helped with better concept visualization; it showed the flow of the class; it helped visual learners, and it made things less intimidating and less overwhelming.

However, 20% of the students reported concept mapping as only somewhat useful, and 2% of students reported it as not useful (Fig. 7a). Those who said concept mapping was not useful noted the following downsides: it is not better than a study sheet and takes too much time; it is only supplemental and not the main study form; it is messy and hard to follow.

Figure 7

Student responses to self-reflection prompts on concept map utility (a), application to other courses (b), the recommendation to peers (c). n = 45.

When asked whether students would be using concept mapping for other courses (Fig. 7b), 84% of the students said yes, 9% said maybe, and 7% said no. When asked if students would be recommending concept maps to their peers (Fig. 7c), 80% responded yes, 16% said maybe, and 4% said no. Those who said “no” or “maybe” reported the following reasons: everyone's study methodologies are vastly different, and one approach that works for them may not work for others. Another reason was that the time and effort required in making concept maps felt significant, and without spending an appropriate amount of time, the concept map became messy.

The instructor agrees that the concept map utility can be decreased if students do not follow the procedure (mentioned in Supplementary Information). Also, it takes time and practice to develop a good concept map. If some students were exposed to it for the first time, they possibly underestimated the time required for mapping concepts.

Peer discussion perhaps also played a critical role in generating metacognition. With peer discussion, one of the reflection prompts (prompt b) asked students—what ideas they got from the group about how to explain the concepts they stumble to describe themselves. Students' self-reflection responses to this question revealed that the peer discussion led to numerous ideas or alternative ways to think about a concept and its relationship with other concepts in the map, boosting the metacognition resulting from the activity. The coded responses of students to this prompt are shown in Table 3 below.

Table 3 Students' self-reflection responses to prompts (a) and (b) coded thematically.

The thematic coding indicated that the topics that the students reported to be difficult were equally diverse for our course and prerequisites. However, a few of the concepts reported as muddy points were from non-prerequisites (Cramer's rule, matrix analysis) that may need to be addressed in future courses to aid students to contextualize content and concepts in the course. The student responses to prompt (b) also indicated how peer discussion benefitted them to contextualize previous knowledge from the current course or from the prerequisite courses with newer mass and energy conservation concepts they were learning in this course.

Additionally, in answering prompt (b), many students also pointed out how peers suggested methods to improve their concept map structure (last row, last column in Table 3). Within this context, concept maps played an important metacognition role for students, making them realize the scope to evolve the representation and functioning of their concept map. In the process, the peers also went through metacognitive exercises to assess whether the concept mapping task requirements were met by their peers and whether the concepts were mapped and explained appropriately.

We observed that the concept mapping exercise with peer discussion required students to demonstrate their cognitive framework through drawing a concept map. At the same time, its oral presentation forced students to articulate their thought processes to peers. This intervention pushed students to demonstrate their basic understanding of concepts, as well as how they related complex, elusive concepts to these basic concepts, thus building upon their prior knowledge.

Student reflections indicated a rich variety of classes that they were considering the application of concept mapping to, such as courses on Biology, Chemistry, Organic Chemistry, Biochemistry, Physics, Mathematics, Circuits, Neurobiology, Physiology and Behaviour, Thermodynamics, Biomedical Signals, Material Engineering, Cellular, and Tissue Engineering, Computer Science, English, Communications, History, Mythology, and Coding.

For example, one student mentioned, [I would use concept maps again] most likely when I am doing the coding. Whenever I code, I make a map of a list of what needs to be coded. The concept map works similarly because I plan out what I need to do, how to approach it, and what method to solve it. It is almost exactly the same with concept mapping.

Some students had even already applied the concept maps to other courses they were taking at the time (Spring 2021).

For example, Mathew reported, I actually ended up using concept mapping in the biology course that I'm taking this quarter (truthfully, I found it more helpful there than in this course). It was useful in a course like Biology which is mostly fact-based because it allowed me to better see the connections between concepts which, at a glance, can seem to be filled with details. Like I mentioned earlier, it was also a low-stress way to wrap up my studying—my studying strategy is to finish the bulk of it a few days earlier and relax a bit before the exam. I see myself using concept-mapping in non-STEM courses or courses that are more detail-oriented.

Diana wrote—I have used something similar to concept mapping before to tie together concepts from different chapters. I have used concept mapping in my chemistry classes before as well as history classes. I have found it especially useful in history classes to identify how an event influences and causes other events further down the road. In physics and chemistry classes, physical concepts like energy, force, velocity, and acceleration are interconnected and can be related on a concept map to get a fuller image.

A couple of students wished they knew about the concept mapping tool earlier. For example, Swara said—I wish I knew more about concept maps in my physics and chemistry classes. All of the topics in each of those classes correlates to one another and explain details from previous chapters. Concept maps would have helped me connect it all together. I will most likely be using concept maps for my final this quarter for physics and organic chemistry, hopefully will get better grades on these exams due to this amazing tool.

Students also mentioned how concept mapping made knowledge accommodation less overwhelming. For example, Jeremy said—I would recommend concept maps to friends. I think that just writing down all the concepts is very important, just so that all of it is on a page. It helps me feel less overwhelmed.

The benefit of interconnections in the concept mapping was revealed by some students. For example, Kevin said—I would recommend concept mapping to peers in other classes because it helps you understand how different concepts are interrelated and how you can use one concept to help understand other concepts.

The use of concept mapping to identify knowledge gaps was mentioned by another student. Maria reported—I would recommend using the concept map to my peers/fellow students because a lot of us struggle with studying and sometimes it is nice to take a step back first and observe what you know well and what you do not. It would help to make a study plan to work on one's weakest points and review the rest.

Interestingly, some students described the technical benefits and how the process of concept mapping itself was fun. For example, Samuel said:

Yes! Of course! [I would recommend concept mapping to peers because] Concept maps are especially useful for students like me who learn better visually, although they can be of benefit to any type of learner who wants an organized and flow-based workspace. They are a powerful study strategy because they help you see the big picture: by starting with higher-level concepts, concept maps help you chunk information based on meaningful connections. Working on problems under a strict time limit with so much information to consider was made possible for me by concept maps and I would want fellow students to succeed as well! There are plenty of benefits to creating a concept map and it can be a lot of fun to just take a break from studying strenuously to sit down and make something too! Especially if it helps you succeed even more!

Conclusion

Through this work, we developed a successful concept mapping intervention that serves as a metacognitive tool to support students' self-assessment of learning in a problem-solving course on mass and energy transfer. Upon developing metacognition-related learning outcomes using concept mapping, we mapped each of the concept map activities to various cognitive process dimensions (remember, understand, apply, analyze, evaluate, create) within the metacognitive knowledge domain in the revised Bloom's taxonomy. This was facilitated in concept mapping exercise by taking students through assessment of a given task's demands, evaluation of one's knowledge and skills, and creation of a plan/an approach to tackle the tasks based on the knowledge and skill assessment.

The concept mapping intervention did not significantly enhance class performance either in-person or online instruction (effect sizes were 0.29 for the 2019 in-person quarter and 0.33 for the 2021 online quarter). However, instructors and students' perceptions reflected that concept mapping served to facilitate metacognition in a problem-solving-based biomedical engineering course both during in-person and online instruction. Furthermore, the benefits of metacognition tools might have additional benefits beyond traditional formal assessments.

Analysis of students' reflections revealed that concept mapping served their metacognitive purpose in this course. Concept mapping was found helpful by students to re-contextualize previous information with the new knowledge taught in the present course. It also helped inform the instructor of the disconnect between previously taken courses. Most students (78%) were optimistic about the usefulness of concept mapping for this course, and 84% were inclined to apply it for a variety of other courses. Overall, the development of the concept maps appeared useful to students for baseline objectives. Although we were unable to establish whether students were better able to address the demands of a task or improve their ability to form a plan, recommendations and ideas from peer comments suggest broadening students’ perspectives in their concept maps that would enhance their understanding to solve problems.

In both the in-person and online instructional modes of this course, building concept maps helped students visually represent their knowledge organization. The peer discussion component facilitated communication of this knowledge organization to other students. Reflection prompts facilitated evaluation of own knowledge organization, making it apparent to oneself and others, and promoted analysis of how new knowledge can be integrated with their existing knowledge structures.

We hope that the other instructors might find this report helpful in applying concept mapping to their classroom to reveal, analyze, enhance their students' knowledge organization, and enhance their metacognitive skills. The self-reflection prompts we designed could be beneficial to those students who focus on the rote-learning method of learning instead of meaningful learning. The concept mapping followed by its peer discussion and reflection activity together can help students recognize how they currently organize their knowledge and improve it in the future. It can also help students analyze if their knowledge of concepts in this course has been built on accurate prior knowledge. In addition to this concept mapping intervention, other helpful tools such as the Metacognitive Awareness Inventory can be simultaneously tapped by the instructors to increase the metacognitive skills.⁶³

Limitations and Future Scope

Problem-solving is considered one of the most important learning activities in engineering education, supporting meaningful learning.¹⁷ To cope with the complexity of knowledge in problem-solving, students need to organize, retain, and apply knowledge to real-life situations.⁸ Metacognitive processes in problem-solving include assessing the requirements of the problem, constructing a solution plan, selecting an appropriate solution strategy, monitoring progress toward the goal, and modifying the solution plan.⁴² We hoped that the concept mapping intervention in this work could be helpful to students in solving problems, but to what extent was it helpful to students in problem-solving, was not specifically studied in this investigation.

Moreover, for metacognition, we wanted our students to be able to (6) assess the demands of a given task, (7) evaluate one’s knowledge and skills, (8) create a plan/an approach to tackle the tasks based on the knowledge and skill assessment; but we did not explicitly assess these outcomes through specific rubrics. Much of the effort in this instance went into the design of concept mapping intervention and its adaption as a metacognitive tool specific to our problem-solving course but future study should aim to develop an assessment rubric and produce a metacognitive skill model specific to this course.

Here we focused on promoting the metacognitive knowledge aspect by mapping the steps in concept mapping intervention to cognitive process dimensions of the revised Bloom's taxonomy. What kind of self-regulatory mechanisms were used by the students and what effect they had on students' problem-solving abilities were also not part of this study. These limitations should be considered while adopting the intervention we developed to other courses.

Our results showed a trend that students who completed concept maps and turned in the self-reflection assignments had a higher average grade. However, this analysis did not control for student GPA. The possibility is not ruled out that high-performing students, who are usually more inclined to do additional assignments, may have appeared to do better in the course, but this may not be directly linked to the concept mapping activity. This correlation remains to be tested. Additionally, this work was performed at a single institution, so the findings are within the context of our institutional setting, and should not be generalized. The sample size in this study was limited to 86 students each for in-person instruction and online instruction. Changes made between the two course implementations (in-person and online) could have also affected the results of this study.

This report also did not consider the effect of concept mapping on students' abilities. In our course, students were exposed to both well-structured as well as ill-structured problems; we did not look at the student performance separately in these two categories of problems when calculating the class performance average. Future research can be directed to find the effect of using concept mapping on students' problem-solving skills in cases of structured vs. ill-structured problems. For example, Kamble et al. have investigated the effect of concept mapping on the performance of mechanical engineering students' problem-solving ability and observed that the concept mapping strategy improved student performance with well-structured problem solving but not in solving ill-structured problems.³⁶ They attributed this to the fact that in solving well-structured problems, the concept mapping strategy helps the student activate the schema, search for a solution, and then implement the solution. Nevertheless, in their study, concept maps provided very little help in solving ill-structured problems because these problems lack an explicit set of rules to solve, have multiple possible solutions, and multiple potential paths.^22,30 Such analysis to find the utility of concept mapping in solving well-structured vs. ill-structured problems would be interesting within biomedical engineering courses involving problem-solving.