Abstract

Keywords

Transfer Students, Performance Analysis, College-to-University, Articulation Agreements, Computer Science, RStudio

Share and Cite:

1. Introduction

The use of college-to-university transfer agreements is not novel and has been utilized in several Canadian provinces as well as in the United States [1] [2], wherein the latter, they estimate that about 45% of baccalaureate degree graduates had previously attended a 2-year college. There has been some concern as to whether students who come to a university from a diploma-granting college will be academically and emotionally prepared to enter university programming on an advanced standing basis [3]. The Department of Computer Science (COIS) at Trent University in Ontario, Canada has been involved with the development of Articulation Agreements¹ with Provincial Colleges since 2001. In 2012, Trent was the first Ontario University to have a System-Wide Pathway agreement in place where a qualified student completing a 3-year diploma in either Computer Programmer Analyst (CPA) or Computer Systems Technology (CST) could enter the B.Sc. (Honours) Computer Science program at Trent [4].

The goal of this research is to compare the academic performance of students who enter the B.Sc. Honours program in a computer-related discipline at Trent University through a formalized articulation agreement relative to the students who come directly from high school. Our study will not just focus on grade-point averages but include metrics such as graduation rates, course failure rates, overall averages, course-level averages, and course-subject averages (COIS, MATH, and Non COIS/MATH). We have explicitly separated computer science courses from math courses as anecdotal evidence has implied that students in computer-related disciplines have not performed as well in math courses as they do in the computer science courses. We are interested in determining if this is a trend, and if so, does it differ between students that enter from college and those that enter directly from high school.

2. Data Used in the Analysis

[1] An Articulation Agreements is a formal agreement between two or more Colleges and Universities documenting the transfer policies for a specific academic program or degree in general.

2In 2008, the Computer Studies/Science Program at Trent University became the Department of Computing and Information Systems and in 2018, the Department of Computer Science.

3Due to low enrollment, the Mathematics and Computer Science (MACS) degree program was discontinued in 2015.

In order to undertake the study, we required anonymized student and course grade data for students entering a computer-related discipline. At Trent, this corresponded to the following degree programs: Computer Science (COIS), Computer Studies/Science (COST/COSC … offered until 2008²), Computing and Physics (COPH), Mathematics and Computing (MACS … offered until 2015³), and Information Systems (INFO). The data was collected for the years 1997 (the year that the Computer Science/Studies Program began to offer a single-major Honours degree) to 2018. This data was provided by the Office of Institutional Planning and Analysis. Table 1 shows a percentage breakdown of the students who were in each of the computer-related degree programs mentioned above. The vast majority (89.4%) were traditional computer science disciplines (COIS, COSC, and COST).

2.1. Student Data

The data collected by the Office of Institutional Planning and Analysis was provided in two files: a student data table and a course-grade data table. The student data table consisted of the following fields: Student Number, Gender, Starting Term, Residency Status (Domestic or International), Program of Study, Agreement Type (if any), Institution Associated with the Agreement, Entry Path (Direct from High School, Transfer with Credits, Articulation Agreement, System-Wide Pathway, and Other), Type of Degree Program (Single, Joint), Graduating Program, Specializations, and Overall GPA. The Other option as listed within the Entry Path field corresponded to students who came to university with no transfer credits: these would include mature students⁴, students from domestic/international colleges/universities, and students from domestic high schools/CEGEP outside of Ontario. The student data table contained 1188 records.

2.2. Course-Grade Data

The course-grade file on the other hand contained the following fields: Student Number, Term Course Taken, Course Number, Course Level, Was Course Replaced, and Grade. The Was Course Replaced field is a Boolean value which indicates whether the student had taken the same course later in their degree program to replace a mark achieved in an earlier attempt at a course. The course-grade data table held just over 28,000 records.

2.3. Data Cleaning

4A mature student is defined as someone who has been out of school (high-school, college, university) for at least 2 years.

The first task was to clean up the data and put it into a form such that it can be properly analyzed. To accomplish this (as well as the data analysis and graph generation), the R programming environment was used as it provides excellent tools for data manipulation, statistical computation, and data visualization [5]. With respect to the student data table, we needed to remove the records for students who had not completed any undergraduate courses for credit as well as removing duplicate entries for students who completed both graduate and undergraduate courses. The student data table was reduced to 1113 records. The course-grade table required significantly more cleaning. One issue was that we had to normalize the data so it was compatible with R (for example, missing values in R are represented by the symbol NA but in the data tables provided missing values were represented with blank entries, the string “N/A” or the string “NULL”). Course-grade records for graduate courses (Course Level = 5000), English as a second language (ESL) courses, courses with no marks (NA), and courses with non-numeric marks (Pass/Fail) had to be removed. The course-grade data table was trimmed by about 2200 records.

3. Analysis of the Student Data

We begin the analysis by focusing on the 1113 records that are part of the student data table with the hopes of achieving a better understanding of the students in the study. The student data were processed using R scripts to first identify how the student arrived to the university (Entry Path), and then based on the Entry Path, how the students were distributed based on gender and residency (domestic versus international). We then moved on to examine more academic attributes for the various Entry Paths such as degree completion rates, degrees completed in a computer-related discipline, and students who changed major (both into and out of a computer-related discipline).

3.1. Entry Path

Table 2 presents a breakdown of the students by the method to which they entered (Direct from High School, Articulation Agreement, Transfer with Credits, and Other) into one of the computer-related disciplines at Trent (COIS, COST, COSC, COPH, MACS, or INFO). We have grouped together the students that enter university from either an Articulation Agreement or the System-Wide Pathway since both groups come through formalized agreements.

From our results, we can observe that the largest group of students were accepted directly from an Ontario High School (37.7%). Interestingly enough, this does not represent the majority as it might with other disciplines within the University. A significant proportion of students (62.3%) coming to university to pursue a computer-related discipline transfer from domestic/international colleges/universities or from domestic high schools outside of Ontario (with or without transfer credits).

3.2. Gender

With respect to gender, the results from Table 3 show that overall, 17.8% of students who came to Trent to study a computer-related from 1997-2018 identify as female. While low, this is on par with other North American Universities but certainly needs to be improved [6]. Results from the table do show an interesting statistic: the largest female/male disparity (13.2% to 86.8%) is with the students coming in from the Articulation Agreements. More investigation into this phenomenon is needed.

3.3. Residency

The residency of the students broken down by entry path is shown in Table 4. The Direct from High School and Articulation Agreement groups are both dominated by domestic students. Students who transfer to Trent with credits are about 2 to 1 in favor of domestic while the Other category is almost 50/50 domestic to international student. This is to be expected given the nature of the entry paths. Overall, 73.1% of students who enter a computer-related discipline at Trent are domestic with 22.8% being international.

3.4. Degree Completion Rates

Table 5 presents the degree completion success for the students across the various entry path groups. We only included those students who had completed (we did not include any students who are currently enrolled in a course).

The most revealing observation seems to be the graduation rate of students from the Articulation Agreement group: 84.5% of students coming from Articulation Agreements graduate with a degree⁵ compared to only 65.0% of students who come directly from an Ontario High School. In addition, it appears from our study that students who enter university with transfer credits also fair better than students directly from high school: a 72.8% graduation rate. The maturity that comes from completing some level of postsecondary schooling before entering university appears to be a benefit.

3.5. Completed Degree in a Computer-Related Discipline

5The degree may not be in a computer-related discipline but this will be discussed in an upcoming sub-section.

We next examine of those students who completed a degree, the proportion who completed a degree in a computer-related discipline. These results are presented in Table 6.

The results again are quite telling with 96.8% of students who come to university through an Articulation Agreement complete their degree within the discipline set out by the agreement. The values for the other three categories are in the mid-to-high 80’s. While not surprising, it is comforting to see that the Articulation Agreement students do tend to remain with the terms of the agreement.

3.6. Students Who Changed Majors

One final metric that we will include in this section is a comparison of the students who declare a computer-related major but do not graduate with a computer-related degree versus the number of students who do not declare a computer-related major but end up graduating with a computer-related degree⁶. These results are show in Table 7 (note that this table does not include those students who declared a computer-related major and graduated with a computer-related degree).

6At Trent University, unless students are in a professional program, they are able to change majors quite easily.

As expected, the results for the Articulation Agreement students show that a very small percentage (less than 5%) change majors. The students in this group are often bound to an agreement so changing majors is more difficult. It is interesting to observe however, that in the other three groups there is a far greater percentage of students who do not declare a computer-related major but end up completing a computer-related degree (in some cases, such as the students from the Direct from High School and other groups, up to three times as much). Overall, our results show that a greater percentage of students seem to switch into a computer-related discipline rather than switch out.

4. Analysis of Course-Grade Data

In this section, we will analyze student performance across the four groups of entry paths. To accomplish this, we joined the student data table with the course-grade data table. This permitted us to create R scripts to examine student performance across entry path groups, disciplines, and course levels.

4.1. Averages across Course-Specific Disciplines

We begin by examining grade averages over the four entry path groups. The data presented here are weighted averages based on the number of credits for each course (i.e., a full course is weighted 1.0 and a half course is weighted 0.5). The information is spread across four tables: Overall Average (Table 8), Average for COIS courses (Table 9), Average for MATH courses (Table 10), and Average for courses which are not COIS or MATH (Table 11).

We can observe from these tables that students that come to university via formalized articulation agreements and students who transfer to university with credits tend to outperform the other two groups across all four categories (in some cases, by 5 - 8 percentage points). Students that come to university directly from high school seem to have the lowest academic performance across all four groups. Not surprisingly, students in computer-related disciplines perform best in COIS courses and have their lowest marks in MATH courses. The results also indicate that the students from Articulation Agreements do tend to struggle a bit more with the MATH courses but not as significantly as students directly out of high school (4.9% drop versus a 7.8% drop). Students who come to university from out of province or as mature students (the Other group) tend to have the most consistent performance across the various discipline-specific areas examined in this study (albeit at a lower academic performance level). In the end, we can infer from these tables that students from Articulation Agreements (and those who transfer with university credits) tend to perform at a higher academic level than students directly out of high school or those who transfer with no academic credits (with the one exception associated with MATH courses).

In Figure 1, we present the academic performance of the four entry path categories graphically across the various course-specific disciplines. We use violin plots (as opposed to box plots) to show the median averages as this type of plot tends to be more informative⁷. Essentially, the results from the violin plots do not affect the overall conclusions when comparing the means to medians. Within each subgraph, different colors are used for each of the entry paths: green for Direct from High School, mauve for Articulation Agreement, pink for Transfer with Credits, and orange for other.

The results indicate that the students who come to university with advanced credits do seem to perform better academically. With respect to the Overall Average, students from the Articulation Agreements tend to perform better than the other three groups. There is only a slight advantage over students with transfer credits but from the violin plots we can observe that the Articulation Agreement group has less variance and fewer outliers then the Transfer Credit group (the plot is wider and does not have a long tail). Both of these groups tend to outperform the students that come directly from High School and the Other group.

7A box plot only shows summary statistics such as median, interquartile ranges, and outliers, while a violin plot shows the full distribution of the data.

With respect to medians, we can see from Figure 1 that the students from Articulation Agreements and students who transfer to university with credits outperform the other two groups with respect to averages in COIS, MATH, and non-MAT/COIS (elective) courses. The group that consistently had the lowest median average across all four course discipline groupings was the students who entered university directly from high school.

An interesting observation from the MATH and non-MATH/COIS courses is the long tails in the violin plots for all entry groups. This implies that all four entry path groups had a number of outliers (averages lower than the bounds defined by a function of the interquartile range) in the MATH and non-MATH/COIS courses in addition to having lower averages in general. This is not surprising however as students tend to perform better academically in their home discipline (COIS courses) which is also supported by the results shown in Figure 1.

The basic conclusion we can draw from Figure 1 is that students that enter university with advanced credits (such as those from Articulation Agreements those that transfer with credits) tend to perform better academically over a wide range of course disciplines.

4.2. Comparison Averages by Course Level and Student Type

We next examine the performance of the various student entry groups by course level to determine if there are any noticeable differences between averages obtained by students in upper-year (300- or 400-level) or lower-year (100- or 200-level) courses spread over the various disciplines (Overall, COIS, MATH, and elective courses). These results are shown in the violin plots given by Figures 2-5. As we did for Figure 1, different colors are used to distinguish the various entry paths: green for Direct from High School, mauve for Articulation Agreement, pink for Transfer with Credits, and orange for other.

We begin by examining the overall course average for the different groups of students for the four different course levels (Figure 2). The first observation is that the students arriving at university through Articulation Agreements or with transfer credits seem to perform better than the other two groups over the four different course levels. This is most apparent in the first (100-level), second (200-level), and third (300-level) year courses. By the time students reach the fourth year courses (400-level), the difference in median averages is quite small. This is to be expected as fourth year courses are quite specialized and students tend to choose them to fit more with their own interests. Another interesting byproduct of this is that median average for all four groups tends to increase by course level: student averages seem to increase with the more advanced (and specialized) courses. This is not surprising as one would expect academic performance to improve with the number of university courses taken as students mature and become more comfortable with university.

In Figure 3, we are focusing on the just the averages for the COIS courses by course level. We observed from Figure 1 that in general, students from this study (pursuing a degree in a computer-related discipline) tended to perform better academically in COIS courses. Now we are interested in determining if the course level affects their academic performance. Similar observations to those in Figure 2 are apparent here: median averages tend to increase by course level, and students from Articulation Agreements or with transfer credits seem to achieve higher averages than the other two groups over all the course levels with larger improvements in the first three COIS course levels (100, 200, and 300). However, as indicated by the tails of the violin plots, students from Articulation Agreements seem to have fewer outliers (low marks) implying their academic performance is more consistent.

With respect to MATH courses, Figure 4 presents violin plots of the student averages of the four different groups of students by level of MATH course. From Figure 1, we know the overall averages for MATH courses are the lowest of the discipline-specific areas but Figure 4 gives us some additional insight. The general trend of students arriving through Articulation Agreements or with transfer credits tending to outperform the other two groups holds for lower-year MATH courses (100- and 200-level). However, for upper-year MATH courses (300- and 400-level), the Articulation Agreement students seem to struggle more: the median averages are the lowest with a significant bulge of outliers in the 300-level MATH courses. While this could be alarming, there are no required upper-year MATH courses required for a degree in Computer Science. These students are taking upper-year MATH courses as degree electives but it is an interesting area of further study.

The final composite graph in this section (Figure 5) presents the academic performance of the four different entry paths groups relative to course level for non-COIS/MATH (or elective) courses. The observations are quite similar to those found in Figure 2 with the students from the Articulation Agreements and those who transferred to university with credits outperforming the students directly from high school and the other group. As the course level increases, the percentage improvement of the Articulation Agreement and transfer students over the other two groups tends to decrease. Even the trends in the number of outliers (low marks) is similar to what was observed in Figure 2.

4.3. Distribution of Letter Grades

Up until this point in the study, we have concentrated our investigation of the students across the four entry paths by focusing on numeric averages with the hope that this would give us some insight into the distribution of the grades. We would now like to examine the grades more closely by presenting the distribution of letters grades for the different entry paths and across the various course groupings.

Figure 6 shows the letter grades achieved by the students overall the courses they completed at university broken down by their entry path.

It is clear from Figure 6 that the students that arrive from the Articulation Agreements and those who transfer with credits, seem to perform quite well relative to the other groups: the darker green area at the top of the stacked box plot in Figure 6 is much larger: about 37% more students from Articulation Agreements or transfer with credits receive A’s compared to students directly from High School. Moreover, when examining the other end of the letter grade spectrum (the orange and green areas at the bottom of Figure 6), about twice as many students directly from High School perform poorly (D’s and F’s) relative to the students from Articulation Agreements or that transfer with credits. The results seem to support that not only do students who arrive university with some postsecondary education perform better, fewer tend to struggle.

With respect to the discipline-specific areas (COIS, MATH, and elective courses), the conclusions about letter grade distributions were quite similar to what was seen in Figure 6 so they are not included but can be found here [7]. The only caveat was the academic performance of the Articulation Agreement

students with respect to MATH courses did not seem to match that of students who transfer to university with credits (but still better than students directly from High School). The implication is that the college-level Math courses may not be adequately preparing Articulation Agreement students for university-level Math courses.

5. Conclusions

The results from our study have led us to conclude that students who enter a computer-related discipline at university through a formalized mechanism (such as an Articulation Agreement or System-Wide Pathway) tend to be academically successful. Compared to students directly out of high school, students from these formalized agreements have higher overall averages, higher averages in their computer-related and elective courses, have a higher degree completion rate (and hence a lower dropout rate), achieve a larger percentage of top marks (A’s), and are less likely to do poorly in their courses (achieve D’s and F’s). These trends also apply to students who enter university with advanced transfer credits. This leads us to conclude that the maturity that comes from completing some postsecondary education seems to be quite beneficial.

Going forward, it is our intention to dig deeper into our student data to examine types of degrees (joint- or single-major, General or Honours), failure rates for specific courses (in both COIS and MATH), and also examine the academic performance broken down by attributes such as gender and residency. We would also like to investigate non-COIS departments at our university that are involved in Articulation Agreements. We are curious to see if the same success we found for students from formalized agreements holds true in other disciplines.

Acknowledgements

We would like to begin by thanking the Ontario Council on Articulation and Transfer for the ONCAT Faculty Fellowship and in particular, Sarah Fuchs at ONCAT for all her help and support. We would also like to thank Hailey Wright, Manager, Community Relations and Articulation at Trent University Durham-GTA for suggesting that we become involved in the project and Kris Lew in the Office of Institutional Planning and Analysis at Trent for handling our NUMEROUS requests for data.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References