Open Educational Resources as a Tool for Educational Equity: Evidence From an Introductory Psychology Class

Introduction

College education is increasingly expensive (Goldrick-Rab, 2016) and this trend negatively affects the well-being of students. For instance, almost half of college students report food insecurity (limited/uncertain availability of quality meals), which is a much higher rate than the general U.S. population (~14.5%, Cady, 2014), and 11–33% of students report housing insecurity (Broton and Goldrick-Rab, 2018). Further, the burden of increasingly high costs of education is experienced disproportionately by marginalized groups. Black students borrow more money for their education and drop out with debt at higher rates than their White counterparts (Huelsman, 2015). First-generation students (i.e., those who are first in their family to attend college) are both more likely to have student loans and to have higher student loans than their continuing-generation peers (Furquim et al., 2017). Finally, Black and Latinx first-generation students are more likely to bear the costs of their education on their own, compared to other groups that are more likely to have assistance from their parents or other support networks (McCabe and Jackson, 2016). Overall, high college costs represent a substantial problem that has important implications for societal equity.

One particular college expense that is rising rapidly, and far-outpacing inflation, is the price of textbooks. From 2006 to 2016, textbook costs rose 87.5% (Bureau of Labor Statistics, 2006) and in 2017, it was reported that students spent an average of $1,260 on textbooks and supplies per year (The College Board, 2017). As a result of these exorbitant costs, students often report engaging in behaviors that could impede their academic progress. For instance, many students (65%) report not buying a textbook because it was too expensive, with the vast majority (94%) of those students reporting that they had concerns that their course performance would suffer as a result of this decision (US Public Interest Research Group Student, 2014). A more recent study found almost half (47.6%) of students occasionally or frequently took fewer courses as a result of high textbook costs, 20.7% withdrew from a course because of high textbook costs, and 45.5% did not register for a specific course as a result of text costs (Florida Virtual Campus, 2016). While we know that high textbook costs lead students to engage in behaviors that may inhibit their academic progress, we do not know if these behaviors are engaged in more frequently for members of marginalized groups. Given that the overall high costs of college disproportionately affect people of color (e.g., Huelsman, 2015; McCabe and Jackson, 2016), it is likely that the negative impact of textbook costs is experienced even more acutely by members of marginalized groups.

One proposed solution to the problems associated with high textbook costs are Open Educational Resources (OER), which are educational materials that are freely available and openly licensed for modifications. OER have been endorsed by people across institutions, with university presidents and faculty overwhelmingly agreeing that colleges should embrace OER (85 and 70%, respectively; Jaschik and Lederman, 2018). A recent study by Seaman and Seaman (2018) revealed that 46% of faculty are now aware of OER, a steady increase from the 34% who were aware of OER 3 years ago. While only 13% of the faculty surveyed reported actually using OER in 2018, this too represents a substantial increase from the 6% who reported using OER in 2016. Further, they found that 22% of faculty who teach introductory courses reported using OER, suggesting wider adoption of OER in lower-level courses, perhaps because more OER are available for these courses.

An estimated 1.2 to 1.6 million students take Introductory Psychology courses each year (Gurung et al., 2016). As such, cost savings associated with this course have the potential to positively impact a large number of students. Using the estimated mean of $100 for a textbook (US Public Interest Research Group Student, 2015), a nationwide adoption of OER in Introductory Psychology could save students $120–160 million per year. However, because Introductory Psychology is often a general education requirement (American Psychological Association, 2014), there is a responsibility to ensure that students do not experience negative outcomes as a result of the use of OER in this course. To this end, there have been several studies comparing performance of students in Introductory Psychology courses using commercial textbooks vs. OER. The results of these studies have been mixed. Engler and Shedlosky-Shoemaker (2018) found no differences in the performance of students' using OER relative to students using a commercial text. In contrast, Hilton and Laman (2012), Clinton (2018), Hardin et al. (2018), and Jhangiani et al. (2018), found better outcomes for students using OER relative to those assigned commercial texts. To our knowledge, only one study found that students who used OER performed worse (on an AP Psychology exam) than those using a commercial textbook (Gurung, 2017).

Many of the studies comparing outcomes of students using OER to those using commercial texts have been conducted under naturalistic conditions. Given the inherent difficulties of conducting research in the classroom, methodological limitations may have contributed to the mixed outcomes of this work (e.g., lack of inferential statistics, Hilton and Laman, 2012; small sample sizes, Grissett and Huffman, 2019). For example, some studies compare classes taught by instructors over multiple semesters (e.g., Hilton and Laman, 2012; Clinton, 2018; Grissett and Huffman, 2019) rather than comparing classes taught by different instructors in the same semester. While the former approach is beneficial because it controls for possible differences in instructor variables (such as experience or enthusiasm), it may confound differences in students' performance across semesters. Moreover, it leaves open the possibility that instructors who are enthusiastic about adopting OER may alter their teaching, consciously or unconsciously, resulting in OER sections performing better for reasons unrelated to the choice of textbook. It is also likely that instructors who are engaged in pedagogical research are invested in being excellent teachers, and thus these individuals may be able to teach students well even when the course materials are subpar. As such, additional studies are needed to examine outcomes of OER in classes where the researcher(s) are not teaching the classes being investigated and where all students are taking the course during the same semester.

To our knowledge only one previous study has examined whether OER disproportionately affects the outcomes of marginalized groups. Specifically, Colvard et al. (2018) found that the use of OER in a range of different college courses improved grades and reduced drop/fail/withdrawal rates for all students. Importantly, students from marginalized populations (i.e., ethnic minorities, students receiving financial aid, and part-time students) experienced larger benefits of OER on these outcomes. It is therefore possible that differences in student populations are contributing to equivocal findings and that OER may help reduce social inequities in the burden of high costs of education.

The present study was conducted to examine perceptions and outcomes of OER, and to explore whether these differ for minority and first-generation students relative to their non-minority, continuing-generation peers. Specifically, we sought to determine the effects of textbook costs on a variety of student behaviors, and whether those effects vary by minority or first-generation status. Second, we wanted to examine whether students assigned an open Introductory Psychology textbook would perform differently than students assigned a commercial Introductory Psychology textbook, as well as whether differences in these outcomes would vary as a function of minority and first-generation status. Finally, we were interested in exploring whether students perceived the two textbooks as equivalent quality and whether they used the two types of textbooks in a similar manner.

Method

Participants

Participants were recruited from 11 sections of Introductory Psychology in the Fall 2018 semester. A total of 774 participants provided informed consent and completed the study. As further detailed in the procedures, six of these sections were assigned an open text and five were assigned a commercial text. Comparisons of the demographic characteristics of these two groups are provided in Table 1. Participants in the two groups (open vs. commercial) additionally differed in the number of courses they were currently taking [t₍₇₆₉₎ = 3.24, p = 0.001)], the number of credits they had completed [t₍₇₆₉₎ = −2.14, p = 0.032)], high school GPA [t₍₇₀₃₎ = 2.45, p = 0.014)], and incoming standardized test scores [t₍₇₀₄₎ = 2.20, p = 0.028)], with participants in the open group taking more courses, earning fewer credits overall, earning a higher high school GPA, and achieving higher scores on standardized tests.

Table 1. Sample characteristics by textbook group.

Consistent with previous research, further comparisons revealed that rates of loans differed significantly by first-generation status with 62% of first-generation students holding loans compared to only 40% of continuing-generation students (χ² = 31.3, p < 0.001). Similarly, rates of student loans also varied by ethnic minority status with 58% of minority students carrying loans compared to 44% of majority students (χ² = 11.73, p = 0.001). The rate of student employment did not differ by ethnic minority status (χ² = 0.49, p = 0.484), but it did differ significantly by first-generation status, with 33.9% of first-generation students working compared to 25.7% of continuing-generation students (χ² = 5.47, p = 0.019).

Procedure

All procedures were deemed exempt from review by the Institutional Review Board. Prior to the semester, graduate student instructors were pseudo-randomly assigned by the third author to use an adaptation of the OpenStax Psychology textbook or the commercial textbook that had been used in the course for the previous 2 years (Scientific American: Psychology, Worth Publishers). The third author trained these instructors in a previous teaching course and was familiar with their teaching background and skills. Group assignment was designed to control for potential confounding and extraneous variables, such as differing levels of instructor experience, section times (i.e., morning vs. afternoon), and days (i.e., M/W/F vs. T/Th).

At the end of the semester, students had the opportunity to complete a survey using Qualtrics (Provo, UT), in exchange for course credit. As detailed in the Materials section, this survey included demographic questions as well as questions on their experience with, and perceptions of, their assigned textbook. After the semester was completed, the Institutional Research office at our university provided information on the participants who gave informed consent and completed the survey, including their final grades in the class, their high school GPAs, and their incoming standardized test scores. Students who did not complete the end-of-semester survey are not included in any analyses as we did not have informed consent or complete data from these students.

Materials

Questions on How Text Costs Affect Student Behavior

Participants reported on whether or not they had engaged in specific behaviors as a result of the costs of textbooks. The options were: bought used copies from the campus bookstore, bought books from a source other than the campus bookstore, bought a digital version of the textbook, rented a printed textbook, rented a digital textbook, used a reserved copy from the campus library, used an inter-library loan, shared a book with a classmate, downloaded a textbook from the internet, stole a textbook, sold a used textbook, didn't use a textbook, or other.

Participants were additionally asked how often they had engaged in a series of other actions as a result of textbook costs. These actions were: taken fewer courses, not registered for a specific course, dropped or withdrawn from a course, earned a poor grade because they could not afford their textbook, not purchased the required textbook. For each of these five items, answers were provided on a scale ranging from 1 (never) to 5 (very often).

Questions on Use of Book

Students were asked whether or not they used the book, how frequently they read the book, and how often they used the book for exam/quiz studying. The latter two questions were answered on a scale ranging from 1 (not at all) to 6 (more than 8 h).

Questions on Perceptions of Book Quality

Questions assessing students' perceptions of the textbook were derived from the Textbook Assessment and Usage Scale (Gurung and Martin, 2011). Specifically, participants rated several aspects of their textbook including the helpfulness, relevance, and explanatory value of their textbook's pictures, graphs, examples, study aids, as well as the textbooks' visual appeal, the clarity of the writing, and the overall book quality, using a scale ranging from 1 (not at all) to 7 (very much).

The survey questions are available here: https://osf.io/ap8hw/.

Data Analysis

The potential effects of first-generation status and ethnic minority status on behaviors related to textbook costs were first analyzed to determine whether textbook costs disproportionally affected students in marginalized groups. To this end, univariate analysis of variance (ANOVA) was used to examine effects of first-generation status, minority status, and their interaction on the total number of alternative behaviors engaged in as a result of textbook costs. A multivariate analysis of variance (MANOVA) was then used to examine how textbook costs affect the frequency of other student behaviors (e.g., dropping/withdrawing from a course due to high textbook costs). Because these outcomes pertained only to behaviors that occurred before the semester in question and therefore could not be influenced by the textbook used in their current course, textbook group was not included as a variable in these analyses. Moreover, participants who reported not knowing if they were a first-generation student or who preferred not to indicate their minority status were excluded.

A hierarchical logistic regression was used to determine whether textbook group (open vs. commercial), ethnic minority status, first-generation status, or interactions between these variables predicted use of the textbook, after controlling for group differences in age, classes currently attempting, credits completed, high school GPA, and standardized test scores (hereinafter referred to as covariates). Only those who reported using their book were included in subsequent analyses.

A multivariate analysis of covariance (MANCOVA) was subsequently used to compare the effects of textbook group, ethnic minority status, first-generation status, and interactions between these variables on a series of questions pertaining to how frequently participants used the book and their perceptions of the book quality, after controlling for the covariates listed above. Follow-up analyses of covariance (ANCOVAs) were used to further clarify significant main effects and interactions, after controlling for the covariates.

Finally, the groups were compared on their overall course performance using ANCOVAs, controlling for covariates. Textbook group, first-generation status, and ethnic minority status were used as independent variables in this set of analyses, due to recent work showing that the effect of OER may be more impactful for students who are typically marginalized in academia (Colvard et al., 2018).

Results

How Text Costs Affect Student Behaviors

A composite score was created to reflect the number of alternate behaviors students reported engaging in as a result of textbook costs (e.g., shared books with a classmate, bought used books). An ANOVA (results in Table 2) revealed a main effect of first-generation status such that first-generation students reported engaging in significantly more behaviors to offset textbook costs than continuing-generation students. There was no significant effect of ethnic minority status nor an interaction between first-generation and ethnic minority status on the number of alternative behaviors reported.

Table 2. Student behaviors by demographic groups.

A MANCOVA (results in Table 2) examining how frequently textbook costs affected other student behaviors (e.g., dropping a course), revealed no significant main effect of ethnic minority status. There was a significant main effect of first-generation status, but this was qualified by a significant interaction between ethnic minority and first-generation status. Follow-up ANOVAs revealed four significant effects of first-generation status and one significant first-generation x ethnic minority interaction. Specifically, as depicted in Figure 1, first-generation students reported engaging in the following behaviors significantly more often as a result of high textbook costs: taking fewer classes, not registering for a specific class, and dropping/withdrawing from a class. As depicted in Figure 2, one item (earned a poor grade because I could not afford to buy the textbook) showed a significant main effect of first-generation status, as well as a significant interaction between first-generation and ethnic minority status. The interaction was probed using independent-samples t-tests that revealed no significant effect of ethnic minority status for continuing-generation students [t₍₅₂₂₎ = 0.63, p = 0.529, d = 0.083]. In contrast, there was a significant effect of ethnic minority status in first-generation students, such that those who are first-generation and an ethnic minority reported that they earn poor grades because of high text costs significantly more often than those who are first-generation and an ethnic majority, [t₍₂₂₂₎ = −2.06, p = 0.040, d = 0.280].

Figure 1. Self-reports of behaviors engaged in because of high textbook costs, by first-generation status. Bars depict means, error bars represent standard errors of the mean, *indicates p < 0.05.

Figure 2. Self-reports of earning a poor grade because students could not afford a textbook, by first-generation and ethnic minority status. Bars depict means, error bars represent standard errors of the mean, * indicates p < 0.05.

Use of Book

A hierarchical logistic regression model using textbook group, ethnic minority status, and first-generation status to predict use of the book after controlling for covariates was not significant, χ² = 8.16, p = 0.418. Interactions between these variables were added in additional steps and further revealed no significant improvement in the model, χ² = 9.53, p = 0.657. All subsequent analyses were conducted using only those students who reported using the book (n = 228 open, n = 169 commercial).

Perceptions of Book Quality

There was no significant multivariate effect of textbook group on students' perceptions of the quality of the book, F_{(17, 264)} = 0.79, p = 0.700, ${\eta }_p^2$ = 0.049. Figure 3 shows the students' overall ratings of the two textbooks, which were not significantly different. There were additionally no significant effects of first-generation [F_{(17, 264)} = 1.152, p = 0.305, ${\eta }_p^2$ = 0.069] or ethnic minority status [F_{(17, 264)} = 1.176, p = 0.284, ${\eta }_p^2$ = 0.070] on perceptions of the quality of the books, nor any interactions between any of these three variables [ethnic minority status x first-generation status, F_{(17, 264)} = 0.691, p = 0.812, ${\eta }_p^2$ = 0.043; textbook group x first-generation status, F_{(17, 264)} = 1.074, p = 0.379, ${\eta }_p^2$ = 0.065; textbook group x ethnic minority status, F_{(17, 264)} = 0.960, p = 0.505, ${\eta }_p^2$ = 0.058; textbook group x first-generation status x ethnic minority status, [F_{(17, 264)} = 1.195, p = 0.268, ${\eta }_p^2$ = 0.071].

Figure 3. Ratings of overall textbook quality across textbook group. Bars depict means, error bars represent standard errors of the mean.

Course Performance

Final course grades were converted from letter (e.g., A, B, C) to a standard GPA scale (A = 4.0, B = 3.0, C = 2.0, etc.). Overall course grades did not differ by textbook group (open vs. commercial) [F_{(1, 327)} = 0.01, p = 0.930, ${\eta }_p^2$ < 0.001], or ethnic minority status [F_{(1, 327)} = 0.22, p = 0.642, ${\eta }_p^2$ = 0.001]. However, the main effect of first-generation status was significant, [F_{(1, 327)} = 7.36, p = 0.007, ${\eta }_p^2$ = 0.022], such that first-generation students performed worse in the course than their continuing-generation peers. There was no interaction between first-generation status and textbook group [F_{(1, 327)} = 0.027, p = 0.870, ${\eta }_p^2$ < 0.001], indicating that the effect of first-generation status on course performance was not related to the textbook used in the course (Figure 4). There was also no significant interaction between ethnic minority status and textbook group [F_{(1, 327)} = 1.480, p = 0.225, ${\eta }_p^2$ = 0.005] nor a significant interaction between all three variables [F_{(1, 327)} = 1.006, p = 0.317, ${\eta }_p^2$ = 0.003] on course outcomes,

Figure 4. Course performance across textbook groups and first-generation status. Bars depict means, error bars represent standard errors of the mean, *indicates p < 0.05.

Discussion

Results from the present study support several conclusions. First, they demonstrate that first-generation and ethnic minority students experience more negative outcomes as a result of textbook costs than their peers. Specifically, first-generation students more often report taking fewer classes, not registering for a specific class, and dropping/withdrawing from a class as a result of high textbook costs, while students who are both first-generation and an ethnic minority more often report earning poor grades because they are unable to afford textbooks. The results further reveal that perceptions of textbook quality and overall course performance did not vary as a function of textbook group, indicating that students perceived the open textbook to be of equal quality to the commercial textbook and performed comparably when an open textbook was used in an Introductory Psychology course.

Many proponents of OER adhere to the “access hypothesis,” which refers to the idea that OER improve performance because more students are able to access the textbook (Grimaldi et al., 2019). Contrary to this hypothesis, we failed to detect differences in the percentage of students who reported using the open vs. commercial textbooks or in the final course outcomes of these two groups. If it is accepted that a core problem addressed by OER is access to textbooks for those who are otherwise unable to afford them, it is also logical that OER would predominantly benefit those students unable to purchase a commercial textbook. Grimaldi et al. (2019) conducted a set of simulation studies to examine this issue. They found that, in a study of 10,000 students where 80% of students could access the book, there was only a 56.5% chance that the null hypotheses (no effect of OER on performance) would be successfully rejected. When the access number increases to 90%, the chances of successfully rejecting the null hypothesis plummets to 19%. This work suggests that tests of the access hypothesis of OER necessitate recognizing that only a small number of students may be helped by these texts. Thus, results may be statistically non-significant, albeit still of immense practical importance for the students positively impacted by access to a free textbook.

Consistent with many studies of OER (e.g., Hilton et al., 2013; Clinton, 2018), students using OER performed comparable to those using a commercial textbook. Although null results are often considered ambiguous, we consider this outcome to be favorable. In the present study, ~600 students across six sections of the course were relieved of a textbook expense of ~$100. Even with a conservative estimate (40% of students buying the book new), this represents a minimum savings of $24,000. If the OER had been adopted across all sections of the course, that cost savings would have approached $50,000 for the semester. In light of that savings benefit, demonstrating that students' outcomes are not harmed by the use of OER is sufficiently compelling to support its use. It is also important to consider these findings in the context of students' wellbeing beyond this one course. Students typically enroll in multiple courses each semester. Cost savings associated with just one course may make the costs associated with other courses more manageable, thus reducing the possibility that students engage in the problematic behaviors associated with textbook costs in their other courses. In this manner, the positive implications of OER use in one course might expand to positively affect students' performance in other courses. This represents an important insight on the broader indirect benefits of OER usage.

Potentially beneficial indirect effects of OER may be especially impactful for marginalized students. Indeed, this study provides further evidence that the rising costs of textbooks is an issue of racial and class equity. A variety of studies show that marginalized students bear a higher burden of college costs than their peers (Huelsman, 2015; McCabe and Jackson, 2016; Furquim et al., 2017). However, this is the first study, to our knowledge, to demonstrate that these inequitable burdens arise, at least partly, as a result of textbook costs. Marginalized students are making a variety of decisions about their academic life based on textbook costs, including which courses to take and whether to drop particular courses. The accumulation of these decisions may contribute to inequalities in first-year retention and graduation rates (Ishitani, 2006; Oseguera et al., 2009). Class-by-class, book-by-book, marginalized students are more vulnerable to financially-driven decisions that can negatively affect their academic progress and outlook, decisions their peers are less likely to encounter. Although the results of the present study only show that marginalized students report engaging in these actions at higher rates, it is also possible that they are contemplating these choices more frequently and more intently then their peers, thus utilizing time and energy that further undermines their academic success.

Textbook costs may also contribute to disparities in outcomes other than grades in a single class. For example, textbooks in fields such as engineering and physics cost far more than textbooks in the humanities (Kopf, 2015). While our data cannot directly speak to this issue, our results provide evidence that marginalized students are negatively affected by high textbook costs more often, including dropping out of, or avoiding altogether, certain classes. Based on this finding, it is possible that textbook costs contribute to the “leaky pipeline” for marginalized students in science, technology, engineering, and math (STEM) fields (Estrada et al., 2016). That is, underrepresented students may drop out of these majors at higher rates due to the high cost of textbooks in many STEM fields. Future studies should evaluate the number of underrepresented students who drop out of STEM majors under baseline commercial textbook conditions, switch the curriculum to OER, then examine whether those drop-out rates decrease. Alternatively, drop-out rates could be compared within a university for STEM programs that do and do not use OER.

There are several limitations to the present study. First, our use of an introductory, general education class means that our sample was comprised primarily of first-year students (62.8%). While not inherently problematic, this does mean that some of our primary dependent measures (e.g., prior behaviors related to high textbook costs) may not be particularly sensitive. The fact that we still found significant differences in these behaviors based on marginalized status suggests that the measures were sensitive enough to reveal differences. Nevertheless, the effects demonstrated here may be even larger than we were able to capture using this sample of predominantly first-year students. Future studies should examine whether these effects are indeed more pronounced in students who have had more time in the college environment. It is also true that effects may be more pronounced in courses where the textbook costs are even higher than the ~$100 for this course on our campus. Future studies should try to replicate our findings in other disciplines and at other universities. Further, we were only able to include students who opted into our end-of-semester survey. As a result, there may be problems with self-selection of participants, and we may have missed important segments of the student population. It is possible that students who are less likely to complete the survey (i.e., those who are failing or otherwise unmotivated) would have provided critical information for our questions of interest. This is additionally true for students who withdraw prior to the end of the semester and thus did not have the opportunity to access the survey.

Conclusions

Overall, we found that students in Introductory Psychology performed equally-well when using OER compared to a traditional commercial textbook. Those using OER also rated the quality of the two textbooks similarly and accessed them at similar rates. Further, we found that students who were first-generation or both first-generation and an ethnic minority reported engaging in more behaviors and suffering more negative outcomes, such as dropping classes or receiving poor grades, because they could not afford the book. Despite the noted limitations, this study supports the conclusion that OER can addresses issues of educational equity while simultaneously maintaining the quality of students' education.

Data Availability Statement

The datasets collected and analyzed for this study can be found on the Open Science Framework at https://osf.io/kxa8v/.

Ethics Statement

The Office of Research Assurances at Washington State University deemed this study exempt from the need for review by the Institutional Review Board. The participants provided their written informed consent to participate in this study.

Author Contributions

AN, CC, and SS contributed to the conception and design of the study. AN organized the data and performed the statistical analyses with help from CC and AN wrote the first draft of the manuscript. All authors contributed to the manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

AN would like to acknowledge the Open Education Group and the Hewlett Foundation for funding her open educational resources fellowship.

References