Introductory programming course: review and future implications

Conception of the review

This step has been conducted to establish the basic understanding of the chosen field through preliminary search and scrutiny of the relevant studies. The searching of studies at this stage was performed manually. It resulted in the identification of 24 records. The preliminary analysis of the IPC studies guided to plan and execute the further steps of this review.

Research questions (RQs)

Keeping in view the objectives of this study, this review addresses the following RQs to synthesize the recent work in IPC and identify the future implications.

RQ1: Which publication sources are the main targets for IPC research and what different types of research are conducted for IPC?

RQ2: What components of IPC are the areas of focus and in what respects those are examined in IPC studies?

RQ3: What contributions can be perceived on the basis of potential benefits or impacts of IPC research in the field?

Search strategy

The search strategy was devised by considering different keywords for searching the relevant records from the digital libraries. These keywords were classified as primary, secondary, and tertiary. The primary set of keywords was established to search the studies, which followed various ways of referring IPC. The secondary set of keywords was devised to find the research articles, which examined different components of IPC, while the tertiary set of keywords was formulated to search the studies, which were specific to student or course. Following is the classification of different keywords that were initially considered for searching the studies:

Primary: Introductory programming, programming fundamentals, programming, CS1

Secondary: learn, teach, assess, content, concept, tool

Tertiary: student, course

The use of logical operators has been emphasized to find the records having the targeted sets of keywords (Marcos-Pablos & García-Peñalvo, 2020). At initial stage of searching the studies, we used two logical operators, AND, and OR, to devise the search strategy. Following was the initial strategy, which involved different classes of keywords and the logical operators to find the relevant papers from the digital libraries: ∀Pimary ∧ (∀Secondary ∨ ∀Tertiary). As a result of executing this approach of searching the studies, a large number of irrelevant records were appeared. Moreover, some databases apply limitations to the number of keywords that can be used in the search string. Hence, we made an optimized selection of keywords to finalize the search string with the intent of targeting more relevant records. From the primary set of keywords, we used “introductory programming”, “programming fundamentals”, and “CS1”, as these were appeared to be the most commonly used terms for representing IPC in literature. Through these keywords, we aimed to search the studies that were related to IPC or the research of programming courses that analyzed specific IPC concepts. From secondary and tertiary sets of keywords, we selected “course” and “student” to target the studies that demonstrated course or student related findings and were focused on different areas of IPC. Moreover, the previous searches showed some irrelevant records that were related to the higher level programming courses like visual or windows programming. Hence, we used another logical operator, NOT, to reduce the possibility of appearing such irrelevant records in the search results. Table 1 enlists the search strings that were specifically applied to each digital repository. Following was the generic search string for searching the relevant papers.

(“Introductory programming” OR “Programming fundamentals” OR “CS1”) AND (“course” OR “student”) NOT (“Visual programming” OR “Windows programming”)

The databases for searching the relevant papers were identified during the preliminary analysis of the literature. Field specific and the most prominent research venues were selected to find the relevant studies. The searches were conducted in the following publication sources: Scopus, ACM Digital Library, IEEE Xplore, SpringerLink, Wiley, Taylor and Francis, MDPI, SAGE, and ERIC. After searching the papers in these sources, a search was performed on google scholar to find any relevant paper that may have left from the prior searches.

Inclusion and exclusion criteria

During the preliminary analysis of papers, some records with inappropriate scopes and focuses (as per the investigating areas of this research) were identified. This further guided to establish the inclusion and exclusion criteria for screening the most relevant records from the search results. Table 2 presents the inclusion and exclusion criteria to shortlist the studies for this review.

Quality appraisal criteria

The quality appraisal of the selected studies was performed on the basis of the following criteria as defined in a previous research (Ouhbi et al., 2015): (a) solution is well defined and possesses methodical potential; (b) conclusion reflects findings; (c) methodology is clear and well defined; and (d) publication source is stable and well-reputed. The studies for criteria ‘a’, ‘b’, and ‘c’ were ranked on the basis of following scores: 1 for fulfilling the respective criteria, 0 for not fulfilling, and 0.5 for partially fulfilling the respective criteria. Criterion ‘d’ was rated by considering the Computer Science Conference rankings (CORE), and the Journal Citation Reports (JCR) lists. Table 3 illustrates the possible ratings for scoring the selected studies for criterion ‘d’.

Teaching

The teaching related aspects were explored by analyzing different pedagogies. The pedagogies can be classified into concept-centric or activity-centric. The concept-centric classification presents those studies in which the pedagogies are analyzed by altering the sequence of teaching programming concepts. The activity-centric pedagogies are explored either by changing the sequence of course related activities or by scrutinizing the impact of conducting one or more activity on students’ learning. In addition, the teaching approaches were investigated through different feedback techniques, which include the dynamic feedback, video-based feedback, and the feedback that was augmented with the visualization of students’ performance through dashboards. Table 7 illustrates brief descriptions of the teaching aspects that were focused in the selected studies and the respective findings.

Summary

A number of different pedagogies were examined in the selected studies. Gamification was deployed as an activity to help students understand programming. Teaching the programming concepts by linking those with real world objects could support learners to conceive the notions behind different programming concepts. Most of the pedagogies were learner-centric in which the learners were required to perform different activities like implementing a specific project and scheduling the learning activities. A few of the pedagogies were teacher-centric through which different teaching techniques were examined. Feedback approaches were mostly tool oriented in which the tools were used to facilitate the feedback delivery processes. The timing of feedback is significant to identify and address the learning gaps. In this context, the dynamic feedback could be useful to provide real time analysis of the learning states and to plan appropriate interventions.

Learning

Learning approaches were further categorized into individual and collaborative learning. The sub-categories of collaborative learning include peer and social learning, while the sub-categories of individual learning include programming, learning style, and learning process. We differentiated the learning styles from learning processes on the basis of providing learners leverage of developing their own pace and style of learning in the case of the former. The learning processes were investigated by examining learners on the basis of specific and defined processes of learning. Table 8 enlists brief descriptions of the learning focused approaches in the selected studies along with the respective findings.

Summary

Learning approaches offered different parameters that can be used to get insight into the behaviors of learners. Assessment and programming data were mainly used to identify learning behaviors. Programming specific behaviors can be examined through students’ programming activities. The parameters to analyze the programming behaviors can dynamically be acquired while solving the programming problems or can be obtained statically through the submitted code. The programming analysis was mainly performed for identifying the parameters that could more accurately be used to predict students’ course performance. In addition, the parameters to analyze the learning behaviors can be acquired by scrutinizing different learning styles and processes. The collaborative learning approaches demonstrated positive impact on students’ learning. Such approaches can be useful for managing large groups of students.

Tool

Tool-centric studies were further classified into IDE and support categories. The classification of IDE includes studies, which analyzed the impacts of using one or more features of IDE on students’ learning. Support tools were further categorized on the basis of the purposes these tools mainly serve or the types of support features that were examined. The sub-categories of support tools include: visualization, which visually demonstrates different aspects of IPC; prediction, which predicts students’ performance; personalized learning, which supports the learning processes on the basis of individual’s learning needs; and feedback, which assists in providing feedback to learners. Table 9 presents brief descriptions of the selected studies that examined different tools and the respective findings of these studies.

Summary

IDE focused studies investigated the enhanced features of different IDEs that could help in solving programming problems. These studies mainly examined the tools for the features, which were more sophisticated than the typical IDEs offer. The support tools were examined to assist one or more components of IPC. The tools were designed to support teaching through performance predictions. Instructors can optimize their teaching efforts through the support features of tools by auto-evaluation of the performance predictions and learning states of the learners. The support tools for learners were mostly used to deliver feedback and assist individual learning processes by evaluating the personalized learning needs of students. Auto-identification of parameters to precisely examine learning can help instructors improve the learning support. It can also be useful to design interventions as per the specific requirements of the learners.

Assessment

The assessment related aspects were further explored by analyzing assessment techniques and assessment material. The sub-categories of assessment techniques include grading and examination. The sub-category of grading includes the studies which examined the techniques of evaluating and ranking students according to their demonstrated understanding levels. The sub-category of examination covers studies in which the effectiveness of some assessment process is investigated. The assessment material is further classified into assessment instruments and assessment items. Table 10 illustrates brief descriptions of the selected studies that were focused on the aspects related to assessments and the associated findings of these studies.

Summary

Self and peer assessment techniques were identified as useful approaches of assessments. The appropriateness of assessment instruments to assess learners is a significant dimension to emphasize the quality of assessment. Some work has been performed to rank and grade learners at various cognitive levels; however, no specific guideline is identified to devise the rubrics for cognitive assessments. Generally, the studies assessed learning without providing information about the specific cognitive levels of learners on which the learners were assessed. The conventional assessment systems can be enhanced through the real-time analysis of learning. For this purpose, a process could be in place to identify the optimal set of parameters that can be used for dynamic analysis of students’ performance.

Content

The content focused studies were categorized into programming language and concepts. Programming language as a sub-category of content includes studies, which either presented comparison of programming languages or analyzed the learning difficulties related to specific programming languages. The sub-category of concepts presents studies in which different choices of constructs were compared to solve a programming problem. It also includes the studies in which specific IPC concepts were analyzed at various cognitive levels of learners. Table 11 enlists brief descriptions of the selected studies that were focused on IPC contents along with the respective findings.

Summary

The cognitive analysis of programming concepts was performed by applying the Bloom’s taxonomy. Analysis of specific concepts was performed for various IPC concepts, such as loops, recursion, iteration, objects, and classes. It appears that the choice of programming languages could impact the learning processes. Hence, a comprehensive study to analyze the effect of teaching different programming languages on students’ performance could be useful in order to identify the most appropriate programming language to teach IPC. In this context, a study has been conducted to evaluate the first programming languages, which presented Java as the most appropriate programming language to teach IPC (Farooq et al., 2014). In addition, utilizing some systematic approach to analyze first programming languages could help in choosing an appropriate programming language out of the available choices. One of the efforts in this direction has been made in which the authors proposed a framework for evaluating the first programming languages (Farooq, Khan & Abid, 2012).

Advice for instructors

As a result of synthesizing the selected studies, some effective practices of conducting IPC have been identified. These practices are listed as advices for instructors. The instructors of IPC can consider these advices to improve the quality of the related aspects of the course.

• Prediction-based intervention design: The difficulty levels of programming concepts tend to rise in the later stages of conducting IPC. Therefore, making use of some performance prediction measures can be beneficial to design and deliver appropriate interventions before teaching the complex concepts of IPC (Omer, Farooq & Abid, 2020).

• Cognitive assessment: Assessing learners on different cognitive levels can be useful for accurate evaluations of learning (Dorodchi, Dehbozorgi & Frevert, 2017). Moreover, assessing the cognitive processes through the exercises enforcing computational thinking can also serve to identify the specific cognitive gaps of leaners (Rojas-López & García-Peñalvo, 2018).

• Gamification as learning approach: Use of games has emerged as an essential technique to support learning in IPC (Malliarakis, Satratzemi & Xinogalos, 2016). This can be an effective approach to develop students’ interests in programming.

• Collaborative learning platforms: Facilitating collaborative environments by establishing the peer learning platforms turned out to be a useful practice in IPC (Ashenafi, Riccardi & Ronchetti, 2015; Echeverría et al., 2017). Tool-supported collaborations could be effective to aid the learning process of novice programmers.

• Use of web-based IDE: Web-based IDE can be useful for programming as it could help students to work remotely on programming problems (España-Boquera et al., 2017; Seeling & Eickholt, 2017). In addition, students’ learning progress can also be conveniently examined by instructors through the submitted programs.

• Tool-assisted feedback: Feedback deliverance can be improved by using tools. The tool-assisted feedback could support the process of identifying the learning gaps of students and help them to understand their personalized learning needs (Fu et al., 2017).

• Grading consistencies and precisions: A study identified huge differences in grading when the same solutions of programming problems were evaluated by different graders (Albluwi, 2018). Applying uniform marking schemes could help to achieve consistencies in grading specifically in case of having more than one graders within or across the terms. Moreover, as a programming problem needs to be solved by applying different concepts, the distribution of marks among various concepts involved in solving a given problem could improve precisions in grading.

• Transparency in assessment: Assessment would be less effective if the students remain unclear about the aspects to be emphasized while solving a given programming problem. Revealing some aspects of assessment criteria to students can help them to differentiate the major and minor aspects that are inquired in assessment items. Another way to address this concern is to provide cues with the problem statements (Doshi, Christian & Trivedi, 2014). However, care must be taken about providing the level of information in the cues, which should not question the assessment process.

• Utilization of tool-interaction data: The tool-interaction data is identified as one of the prominent parameters to gage students’ involvements in the course (Chaweewan et al., 2018). Analysis of tool-interaction data could be useful to understand students’ learning behaviors. Similarly, the analysis of programming data related to the use of tools can also help to get insight into the individual progression of learning.

Open research issues

The analysis of the findings revealed some significant dimensions that can be considered for future research of IPC. These are listed as open research issues of IPC.

• Robustness of IPC research: The findings of most of the selected studies were based on particular cohorts of students. Rare efforts are observed to examine the robustness of existing work by replicating the same scenarios on different groups of learners. A study performed replication analysis of the research conducted in programming courses and found differences in the results (Ihantola et al., 2015). This gap questions the applicability of studies on similar scenarios and indicates the need of investigating the robustness of IPC research.

• Code testing: Generally, the IPC contents do not include the topic of testing. Consequently, students would not be able to formally verify or validate their programs. The findings of the studies, which analyzed learning approaches through programming related parameters like errors (McCall & Kölling, 2019) and test cases passed (Ureel & Wallace, 2015), could have been influenced due to this potential gap in the IPC contents. A recent research explored different ways of teaching software testing in IPC (Scatalon, Garcia & Barbosa, 2020); however, the identification of appropriate level of cognition to teach the concept of code testing in IPC is an area that still needs to be examined.

• Course structuring: Assessment of learning at initial stages of conducting IPC, and analyzing the impacts of the outcomes of assessment on forthcoming stages can help in early identification of learning gaps. To follow such practice, IPC needs to be structured by linking various stages of course covering different concepts. This can be done by using the technique of concept mapping. A study analyzed the use of concept mapping in computer science courses and identified the increase in diversity of its use in programming courses (Santos et al., 2017). A recent work proposed the use of concept mapping technique to examine cognitive performance on higher level programming concepts (Omer, Farooq & Abid, 2020). However, the use of concept mapping can further be evaluated from various dimensions like investigating the cognitive learning patterns, differentiating the cognitive patterns of specific groups of learners, and examining the threshold concepts.

• Behavioral assessment: The behavioral assessment supports precision teaching, which is a constructional approach to solve behavioral and learning problems (Evans, Bulla & Kieta, 2021). IPC generates huge volumes of data in the form of submitted programs, and tool interactions of students, which can be utilized for assessing the learning behaviors of IPC students. The behavioral assessment can also be used for developing and maintaining the programming profiles in order to track the behavioral progressions of novice programmers.

• Gamification: An increasing trend of using gamification, in computer science education, has been observed in recent years (Ahmad et al., 2020). Gamification with mobile-assisted learning can enhance the intrinsic motivation of learners (Ishaq et al., 2021). A study suggests that the use of games can improve students’ engagements in programming (Rojas-López et al., 2019). However, rare work is observed to evaluate how gamification can precisely be used to support teaching, learning, and assessment of specific IPC concepts. Moreover, designing and implementing games as learning objects with respect to certain cognitive levels of learners is an area that still needs attention of the community.