Sage Journals: Discover world-class research

Abstract

In the current study we examined students' course performance, perceptions, and self-reported use of an open textbook compared to a traditional publisher's textbook in an introductory psychology course. Sixty students from two course sections used either an open textbook or a traditional textbook. To minimize the effect of confounding variables, each section was taught by the same instructor, at the same time of day, using identical in-class materials, syllabi, and sequencing, on alternating days of the week. Course performance was measured by analyzing exam scores and final course grades. A 22-item survey was utilized at the end of the term to measure students' textbook perceptions and use. Findings revealed no significant differences in student course performance or textbook use across the two sections. Participants in both groups perceived cost, weight, and convenience as the biggest advantages of an open textbook and perceived ease of reading, convenience, ability to highlight and take notes, ability to quickly find a topic, and ability to keep as a reference as the biggest advantages of a traditional textbook. Finally, when asked to select which type of textbook they preferred, students in each section selected the textbook they were using. The implications of these findings are discussed.

Keywords

Introduction to Psychology open educational resources open textbooks student perceptions of textbooks student performance student textbook use traditional textbooks

Many instructors, including those in psychology, are making the shift to open educational resources (OER), including open textbooks, because of their perceived cost savings for students, flexibility, and increased access to course material (Hilton, 2016). This shift increases the need to address questions regarding the efficacy and perceptions of the texts among students. A growing body of research has examined these qualities of open resources, including cost, outcomes, use, and perceptions, also known as the COUP model (Bliss, Robinson, Hilton, & Wiley, 2013). Findings from these studies suggest that there are few, if any differences, between outcomes (e.g., Fischer, Hilton, Robinson, & Wiley, 2015; Hilton, 2016) and perceptions (e.g., Bliss, Hilton, Wiley, & Thanos, 2013; Hilton, 2016; Hilton, Gaudet, Clark, Robinson, & Wiley, 2013) of open textbooks as compared to traditional textbooks.

The current study examines students' course performance, perceptions, and self-reported use of an open textbook compared to a traditional publisher's textbook in an introductory psychology course. It is nearly impossible to attain true experimental conditions in a classroom setting, but our aim was to control several common extraneous variables that can lessen internal validity, including instructor, semester, and time of course. We compared three outcome variables–course performance, perceptions, and self-reported textbook use–between two introductory psychology courses–one that used an open textbook and one that used a traditional textbook. Both courses were taught by the same instructor, in the same room, at the same time of day, using identical in-class materials, syllabi and sequencing, during the same semester. The same instructional techniques and activities were also used for both course sections. By controlling these factors, we hoped to gain a better understanding of how student performance, perceptions, and use might differ according to the textbook being used.

Literature Review

One of the main drivers of open textbook adoption is the rise of textbook costs. College students spend an average of $1200 per year on textbooks and supplies (Ma, Baum, Pender, & Bell, 2015), a cost that exceeds other education costs (e.g., tuition, housing) over the past decade (Bureau of Labor Statistics, 2016). To reduce cost, instructors have adopted open (free) materials. These materials are typically available under a Creative Commons license, which allow users to retain, reuse, revise, remix, and redistribute the materials (Wiley, n.d.). Free materials have unquestionably saved students money, but do they provide an effective alternative to traditional textbooks in terms of student performance?

Efficacy of Open Materials

Research addressing the efficacy of open materials on student course performance has been mixed, but most publications in this area suggests that open materials typically yield equivalent, if not in some cases, better, student course performance outcomes than traditional textbooks (Hilton, 2016). These results span multiple disciplines, including psychology (Clinton, 2018; Fischer et al., 2015; Hilton & Laman, 2012), history (Grewe & Davis, 2017), business (Feldstein, Martin, Hudson, Hilton III, & Wiley, 2012), biology (Fischer et al., 2015), math (Chiorescu, 2017; Fischer et al., 2015; Hilton et al., 2013; Pawlyshyn, Braddlee, Casper, & Miller, 2013), computer networks (Gil, Candeles, Jara, Garcia, & Torres, 2013), statistics (Bowen, Chingos, Lack, & Nygren, 2012), physics (Hendricks, Reinsberg, & Rieger, 2017), and chemistry (Allen, Guzman-Alvarez, Molinaro, & Larsen, 2015).

In psychology, Hilton and Laman (2012) found that 690 students using an online, open psychology textbook had higher course grades, final exam grades, and higher retention rates than students who did not use an open text in the previous semester. Fischer et al. (2015) compared performance across 15 courses, four of which were psychology, in a multi-institutional study. They measured course completion, passing grades (C or better), and course grades across courses that used OER and ones that used a traditional textbook. Of these measured outcomes in the psychology courses, students who used an open text in one course (Psych 100) had a higher course grade than students who used a traditional text. The other results did not reveal any other group differences.

Although these studies have illustrated the impact OER have on students, to our knowledge none have controlled for instructor, instructional strategies, semester, room, course, and time of day in a single study. We controlled these factors within the present study in the hope of better understanding the impact of OER on its users.

Student Perceptions of Open Materials

Research on student perceptions of open compared to traditional learning materials reveal similar results–students largely view OER similarly, or at times more positively, than a traditional textbook (e.g., Bliss et al., 2013a; Bliss et al., 2013b; Delimont, Turtle, Bennett, Adhikari, & Lindshield, 2016; Feldstein et al., 2012; Gil et al., 2013; Hendricks et al., 2017; Hilton et al., 2013; Illowsky, Hilton, Whiting, & Ackerman, 2016; Ozdemir & Hendricks, 2017; Watson, Domizi, & Clouser, 2017). Findings regarding students' positive perceptions of OER have also been found among students taking online classes (Lindshield & Adhikari, 2013).

As most OER are online, including those used in the current study, it is important to also consider students' perceptions of online materials as compared to print. The majority of previous research in this area suggests that students prefer print over online materials (e.g., McGowan, Stephens, & West, 2009; Shepperd, Grace, & Koch, 2008; Terpend, Gattiker, & Lowe, 2014; Vernon, 2006). Despite this preference, research suggests using online materials does not translate into lower course performance (Rockinson-Szapkiw, Courduff, Carter, & Bennett, 2013; Shepperd et al., 2008; Siebenbruner, 2011; Terpend et al., 2014).

Student Use of Open Materials

In terms of students using and accessing open materials, findings have been mixed. Bliss et al. (2013a; 2013b) found no differences between students' use of open textbooks and traditional textbooks. Conversely, Lindshield and Adhikari (2013) found that students in online and face-to-face courses reported using their open text more often than traditional texts. Likewise, Allen et al. (2015) found that students spent more time using the free online chemistry site, ChemWiki, than traditional learning materials. In that study, Allen and colleagues acknowledged that the reason for increased use of OER is unclear. Indeed, there are many factors that influence student use of materials, and although being “open” may seemingly increase student use because of free, open access to the materials, existing research is not clear whether this is the case. The current study does not necessarily address the core of the question of how OER might increase use, but it does further explore the use of an open textbook as compared to a traditional textbook.

Current Study

Research to-date suggests that open materials yield similar course outcomes and are generally deemed to possess positive characteristics when compared to traditional course materials. Several of these studies, however, did not closely control several factors, including semester, course instructor, and time of day, which may have influenced the outcome of the studies (Griggs & Jackson, 2017). The present study attempts to minimize these factors by examining outcome variables across two course sections taught by the same instructor, using the same instructional strategies, in a single semester, in the same room, at the same time of day, using identical in-class materials, syllabi and sequencing, on alternating days. Our specific research questions were as follows:

RQ1: Are there differences in student performance between students who use an open textbook and those who use a traditional textbook?

RQ2: What are student perceptions of an open textbook compared to a traditional textbook?

RQ3: Are there differences in student use of an open textbook compared to a traditional textbook?

We conducted descriptive and inferential analyses to test each question. Because the research on each of these areas is mixed, we did not have any specific hypotheses. By addressing all three questions through a quasi-experimental design, we sought a better understanding of how open textbooks compare to traditional textbooks in an introductory psychology course.

Method

Participants

Sixty students from two introductory psychology course sections at a small, public university in the southeastern United States participated in this study (n[open text users] = 31; n[traditional text users] = 29). This included 23 males and 37 females, with an average age of 19.34 years (SD = 1.84 years) across the two course sections. All students provided informed consent for their completion of the perceptions survey, and IRB approval was acquired.

We did not conduct any pre-test to measure students' pre-existing knowledge about psychology. Students were largely from first-year except for a sophomore (n = 1), a junior (n = 1), a senior (n = 1), and two students who did not report their class standing. Students represented more than ten majors from across campus.

Materials

Textbooks

Students in the open-textbook course section used the Introduction to Psychology: Version 1.0 (Stangor, 2010) textbook. This textbook was selected because it was the only open introductory-level psychology textbook available when textbooks were selected for the course. Students could download the textbook for free in a .pdf format from the Saylor Foundation website (http://saylor.org). A link to the textbook was also provided in the course management platform, Desire2Learn. The instructor did not make any modifications to the textbook.

Students in the traditional publisher's textbook course section used Psychology in Everyday Life, 3rd edition (Myers & DeWall, 2014). This text was selected because the instructor had used this book in previous semesters, and it was used by other faculty in the department. This book retailed for approximately US$100 at the school bookstore. Students in both course sections were required to have their respective textbook.

Survey

We administered a 22-item survey to measure students' perceptions (including preferences, perceived advantages, and characteristics) and self-reported use of the open and traditional textbook. The survey was modified for each course section, so that students answered questions regarding the material they used. To clarify the text in the open-textbook group, that survey used the phrase “free, online textbook,” whereas the phrase “traditional paper textbook” was used in the traditional publisher's textbook survey. Except for Questions 20 and 21, which were about the perceived advantages of each type of text, and Question 22 about textbook preference, students' ratings were only for the textbook they were using.

Questions 1–4 pertained to students' ideas about the textbook they were using (e.g., “I would like to have a free online textbook to use in this class in addition to the traditional paper textbook”) and were scored on a 7-point Likert scale (1- strongly disagree; 7- strongly agree). Questions 5–17 pertained to students' perceptions of the textbook quality (“Rate the overall level of quality of the textbook you used in this course”) and were also scored on a 7-point Likert scale (1- very poor quality; 7-very good quality). Questions 18 and 19 asked students about their use of the textbook (“Compared to your experience with textbooks in other courses, how often do you use the textbook in this course?” and “How frequently do you use the textbook in this course?”). Both were scored on a 7-point Likert scale (respectively, 1- much less often, 7- much more often; and 1- never, 7- more than three times a week). Question 20 asked students “What do you see as the advantage(s) of a free online textbook? (check all that apply).” Question 21 asked students “What do you see as the advantage(s) of a traditional paper textbook? (check all that apply).” The advantages listed, for both Questions 20 and 21, were “cost,” “ease of reading,” “weight,” “convenience,” “ability to highlight and take notes,” “ability to quickly find a topic,” “can keep it as a reference book for future use,” and “other (please specify).” Students could select more than one advantage for each type of text. For Question 22 (“If given the choice between a traditional paper text and a free, online text, which would you choose?”), students were asked to choose between either a “free, online text” or “traditional paper text.”

Except for Question 22, which was developed by the researchers, all other questions were adapted from textbook scales used in previous research regarding textbook quality, use, and satisfaction (Gurung & Landrum, 2012; Lindshield & Adhikari, 2013; McGowan et al., 2009), including the Textbook Assessment and Usage Scale (TAUS; Gurung & Martin, 2011).

Procedure

Textbook implementation

Textbook implementation and data collection were carried out in spring 2015. The two course sections were randomly assigned an open or traditional publisher's textbook. As open resources had not been adopted within the department before, this was an opportunity for faculty in the department to better understand the implementation process, the strengths and weaknesses of using an open textbook, and to contribute to the extant body of literature on open educational resources in higher education.

Students using the open textbook could access the textbook online at no cost. The instructor guided the students using the open textbook through the steps of accessing the textbook. Students could access the textbook from a hyperlink in the course syllabus and through the course learning management system.

Students using the traditional textbook were allowed to purchase, rent, or use a free-copy that was made available at the school library (for two-hour intervals). The same instructor (one of the authors) taught both course sections in the same room, and each course was offered at the same time but on alternating days of the week during the same semester. Thus, instructor, location, time, and semester were controlled across groups. Course sequencing and instructional strategies were held constant across each course section, with the instructor using the same slide presentations, lecture notes, and class activities in both sections.

Student course performance

To measure student course performance, we analyzed five exam scores, using identical exams in each course, as well as final course grades. Course grades comprised the five exams (worth 76% of the course grade), a final paper (worth 19%), and a research participation requirement (worth 5%). Exam questions were created by the instructor and drawn from the test-bank of the traditional text. All questions were textbook agnostic, meaning they only covered information that was covered in each textbook. These included definitions of terms that were covered in each text and everyday examples that were not covered specifically in either text. Exams were not cumulative and were given on consecutive days.

Student perceptions and use

We administered surveys to students in each course section at the end of the semester to capture students' perceptions of open and traditional textbooks. The surveys were printed, stapled, and given to students to complete by hand during class. The surveys took approximately 20 minutes to complete. Students earned a small amount of extra credit (5 points, or approximately 1% of the total possible course points) for participating. Overall participant response rate was 90.91%, with group response rates as 87.88% for the traditional textbook section (n = 29) and 93.94% (n = 31) for the open textbook section.

Results

Student Performance

Our first research question was whether there are differences in student performance between those who use an open textbook and those who use a traditional textbook. We first compared the total number of exam points across the two groups. The same five exams were given to students in both courses. Each exam was worth 80 points, so students could earn up to 400 exam points.

Open-textbook users earned an average of 273.26 total exam points (SD = 79.37) compared to 295.91 total exam points (SD = 67.68) among traditional textbook users. An independent group t-test revealed that this difference was not statistically significant, t(64) = −1.329, p = .188.

Next, we compared performance across the two course sections on each of the five exams (Table 1). No differences were found between the two groups on any of the exams.

Table 1.

Student Performance on Five Exams.

Exam	Group	N	Mean	SD	t	df	p
1	OER Trad	32 32	62.17 64.39	8.84 7.71	−1.07	62	.289
2	OER Trad	31 31	60.90 64.98	10.33 9.75	−1.60	60	.115
3	OER Trad	32 33	49.16 52.73	11.90 12.93	−1.59	63	.251
4	OER Trad	30 31	61.30 62.58	9.24 8.71	−.557	59	.580
5	OER Trad	27 30	66.96 67.01	9.09 8.05	−.019	55	.985

Note. Mean scores were out of a possible 80 points.

Last, we compared final grades, which included the five exams (worth 76% of the final grade), a paper (worth 19%), and a research requirement (worth 5%). Traditional textbook users' final grades were M = 73.72; SD = 18, and open textbook users' final grades were M = 70.82; SD = 21. These differences were not statistically significant, t(62) = −.611, p = .544.

Student Perceptions of Textbooks

Our second research question was “what are student perceptions of an open textbook compared to a traditional textbook?” To address this question, we first examined students' ideas about the textbook (Questions 1–4), and perceptions of the characteristics of the textbook they were using (Questions 5–17). Next, we asked about students' textbook use (Questions 18–19). Third, students listed advantages of both open and traditional textbooks (Questions 20 and 21). The last question (Question 22) asked students which type of text they preferred most (open or traditional). Results for each of these questions are listed below.

Perceptions of textbook quality. First, we wanted to know about students' perceptions of the text they were using for each of the following factors: ideas about use, cost, satisfaction, ease of use, visual appeal, engagement with the text, ability to understand the text, examples, study aids, feelings of relaxedness, and overall quality. Student perceptions of textbook characteristics were measured by 17 questions (Questions 1–17) about the textbook they were using. To analyze group differences between students using the open text and those using the traditional text, we conducted descriptive analyses (means, standard deviations, standard error, and confidence intervals), as well as inferential statistics (t-tests).

Students using the open textbook were more likely to like the idea of their text (Question 1), t(58) = −3.39, p = .001, d = .868, 95% CI [−1.62, −.42], prefer using their text (Question 2), t(58) = −2.48, p = .016, d = .634, 95% CI [−2.18, −.23], and like the idea of (not) buying their textbook (Question 3), t(58) = −5.22, p < .000, d = 1.336, 95% CI [−3.06, −1.36]. For Question 4, students in traditional textbook group were more likely to want a “free, online” textbook in addition to the traditional textbook they were using compared to students in the open textbook group, who were less likely to want a traditional textbook in addition to the text they were using, t(58) = 2.50, p = .015, d = .649, 95% CI [.25, 2.28]. Finally, students in the traditional textbook group were more likely to rate their textbook as visually appealing (Question 15) than students in the open textbook group, t(58) = 3.93, p < .000, d = 1.025, 95% CI [.58, 1.77].

Perceived textbook advantages. To further assess students' perceptions of textbooks, we asked students to select the perceived advantages for each type of text (open and traditional). Whereas the other survey questions examined students' perceptions of only the textbook they were using, questions on perceived advantages (Questions 20 and 21) asked students about the perceived advantages of both types of textbooks (“What do you see as the advantage(s) of a traditional paper/free online textbook? (check all that apply)”). The advantages listed were “cost,” “ease of reading,” “weight,” “convenience,” “ability to highlight and take notes,” “ability to quickly find a topic,” “can keep it as a reference book for future use,” and “other (please specify).” Students could select more than one advantage for each textbook.

Figure 1 illustrates students' perceived advantages of an open and traditional textbook. Students in both courses (n[traditional textbook users] = 29; n[open textbook users] = 31) perceived cost, weight, and convenience as the biggest advantages of an open text. Students perceived ease of reading, convenience, ability to highlight and take notes, ability to quickly find a topic, and ability to keep as a reference the biggest advantages of traditional textbooks.

Figure 1.

Students' perceived advantages of open and traditional textbooks.

Student textbook preference. Responses on Question 22 (“If given the choice between a traditional paper text and a free, online text, which would you choose?”) revealed that 69% (n = 20) of traditional textbook users preferred a traditional paper textbook, and 70% (n = 21) of students who used the open text preferred the open textbook. A non-parametric chi-square analysis revealed that these differences were statistically significant, χ2(1) = 8.96, p = .003. Thus, students were most likely to prefer the textbook they were assigned.

Textbook Use

Our third and final research question asked, are there differences in student self-reported use of an open textbook compared to a traditional textbook? We measured this through two questions on our survey (Questions 18 and 19). Question 18 asked, “Compared to your experience with textbooks in other courses, how often do you use the textbook in this course?” It was scored on a seven-point scale (1- much less often; 7- much more often). Inferential t-tests were used to determine potential group differences. Traditional textbook users scored a mean of 3.79 (SD = 2.45), and open textbook users scored a mean of 3.55 (SD = 1.80). These differences were not significant, t(64) = 0.45, p = .652, 95% CI [−1.09, .60] (Table 2).

Table 2.

Perceptions of Traditional Textbooks among Traditional Textbook Users (n = 29) and Open Textbooks among Open Textbook Users (n = 31).

	Question	Condition	M	SD	SE	t	df	p	Lower 95% CI	Upper 95% CI
Q1	I like the idea of a (free, online/ traditional) textbook.	Traditional	5.69	1.51	.28	−3.39	58	.001	−1.62	−.42
Q1	I like the idea of a (free, online/ traditional) textbook.	Open	6.71	0.69	.12	−3.39	58	.001	−1.62	−.42
Q2	I prefer using a (free, online/ traditional) textbook.	Traditional	4.83	2.16	.40	−2.48	58	.016	−2.18	−.23
Q2	I prefer using a (free, online/ traditional) textbook.	Open	6.03	1.58	.28	−2.48	58	.016	−2.18	−.23
Q3	I like the idea of buying a (free, online/ traditional) textbook.	Traditional	4.21	2.02	.38	−5.22	58	.000	−3.06	−1.36
Q3		Open	6.42	1.18	.21	−5.22	58	.000	−3.06	−1.36
Q4	I would like to have a (free, online/ traditional) textbook in this class in addition to the (free, online/ traditional) textbook.	Traditional	5.66	1.65	.31	2.50	58	.015	.25	2.28
Q4		Open	4.39	2.22	.40	2.50	58	.015	.25	2.28
Q5	How satisfied are you with the textbook?	Traditional	6.00	1.22	.23	−.45	58	.654	−.70	.45
Q5	How satisfied are you with the textbook?	Open	6.13	0.99	.18	−.45	58	.654	−.70	.45
Q6	How would you rate the ease of using the textbook?	Traditional	6.00	1.25	.23	.30	58	.763	−.54	.74
Q6	How would you rate the ease of using the textbook?	Open	5.90	1.22	.22	.30	58	.763	−.54	.74
Q7	How engaging/interesting is the writing of the textbook?	Traditional	5.38	1.05	.19	−.49	58	.628	−.70	.43
Q7	How engaging/interesting is the writing of the textbook?	Open	5.52	1.12	.20	−.49	58	.628	−.70	.43
Q8	How understandable/clear is the writing of the textbook?	Traditional	6.31	0.93	.17	−.49	58	.629	−.56	.34
Q8	How understandable/clear is the writing of the textbook?	Open	6.42	0.81	.14	−.49	58	.629	−.56	.34
Q9	How would you rate the everyday life examples used to explain the material in the textbook?	Traditional	6.00	1.07	.20	−1.69	58	.096	−.84	.07
Q9		Open	6.39	0.67	.12	−1.69	58	.096	−.84	.07
Q10	How relevant are the everyday life examples in the textbook?	Traditional	5.90	0.86	.16	−1.17	58	.246	−.63	.16
Q10		Open	6.13	0.67	.12	−1.17	58	.246	−.63	.16
Q11	How adequate are the number of study aids (e.g., review questions) in the textbook?	Traditional	5.76	1.09	.20	−.63	58	.533	−.74	.39
Q11		Open	5.94	1.09	.20	−.63	58	.533	−.74	.39
Q12	How helpful are the study aids in the textbook?	Traditional	5.69	1.26	.23	−.41	58	.684	−.69	.46
Q12	How helpful are the study aids in the textbook?	Open	5.81	0.95	.17	−.41	58	.684	−.69	.46
Q13	How relaxed or unrelaxed did the textbook make you feel while you were reading it?	Traditional	4.97	1.32	.25	−1.13	58	.265	−1.08	.30
Q13		Open	5.35	1.36	.24	−1.13	58	.265	−1.08	.30
Q14	How critically did the textbook make you think while you were reading it?	Traditional	5.45	1.21	.21	−.87	58	.387	−.75	.30
Q14		Open	5.68	0.91	.16	−.87	58	.387	−.75	.30
Q15	Overall, how would you rate the visual appeal of the textbook (e.g., layout, colors, cover)?	Traditional	6.21	0.94	.17	3.93	58	.000	.58	1.77
Q15		Open	5.03	1.33	.24	3.93	58	.000	.58	1.77
Q16	Based on the colors, layout, font, figures, writing quality, and general look of the textbook, how would you rate your ability to concentrate on the contents of the textbook?	Traditional	6.10	0.82	.15	1.62	58	.111	−.11	1.03
Q16		Open	5.65	1.31	.23	1.62	58	.111	−.11	1.03
Q17	Rate the overall quality of the textbook you used in this course.	Traditional	6.31	0.97	.18	.08	58	.934	−.46	.50
Q17		Open	6.29	0.90	.16	.08	58	.934	−.46	.50
Q18	Compared to your experiences in other courses, how often do you use the textbook in this course?	Traditional	4.21	1.45	.27	−.58	58	.566	−1.09	.60
Q18		Open	4.45	1.80	.32	−.58	58	.566	−1.09	.60
Q19	How frequently do you use the textbook in this course?	Traditional	4.69	1.58	.29	−.39	58	.695	−.91	.61
Q19	How frequently do you use the textbook in this course?	Open	4.84	1.34	.24	−.39	58	.695	−.91	.61

Note. All responses were scored on 7-point Likert scales with higher values representing stronger responses (e.g., strongly agree, very helpful, very often).

Question 19 asked, “How frequently do you use the textbook in this course?” It was scored on a seven-point scale (1- never; 7- more than three times a week). Traditional textbook users scored a mean of 3.31 (SD = 1.58), and open textbook users scored a mean of 3.16 (SD = 1.34). These differences were not significant, t(64) = 0.42, p = .680, 95% CI [−.91, 61] (Table 2). Essentially, it appears that regardless of what textbook condition the students were in, it did not influence their reports of textbook use.

Discussion

Overall, our study provided a much-needed empirical look at student perceptions, self-reported use, and performance when using an open psychology textbook compared to a traditional, publisher's textbook through a quasi-experimental approach. In our study, we controlled for instructor, instructional strategies, semester, room, time of day, syllabi, sequencing of in-class content, and course to maximize internal validity. This was the first study to our knowledge that took all of these factors into consideration within a single study.

Our findings revealed no significant differences between groups in terms of performance and self-reported textbook use. This finding is consistent with previous research, which demonstrates similar student performance, regardless of the modality of the text (e.g., Allen et al., 2015; Bowen et al., 2012; Chiorescu, 2017; Clinton, 2018; Feldstein et al., 2012; Fischer et al., 2015; Gil et al., 2013; Grewe & Davis, 2017; Hendricks et al., 2017; Hilton & Laman, 2012; Hilton et al., 2013; Pawlyshyn et al., 2013). Despite the instructional strategies being tailored more to the traditional than the open textbook, performance across the two sections did not differ as the instructor had used the traditional textbook in previous semesters. This illustrates that even at a potential instructional disadvantage, students using the open textbook did as well as those using the traditional textbook.

Open textbook users also tended to like the idea of an open textbook more than traditional textbook users liked the idea of a traditional textbook. Traditional textbook users also wanted a “free, online textbook” in addition to the traditional paper one they were using. These findings were in line with prior research that suggests students tend to prefer open textbooks or at least view them similarly in relation to traditional textbooks (Bliss et al., 2013a; Bliss et al., 2013b; Delimont et al., 2016; Feldstein et al., 2012; Gil et al., 2013; Hendricks et al., 2017; Hilton et al., 2013; Illowsky et al., 2016; Ozdemir & Hendricks, 2017; Watson et al., 2017).

Previous research suggests that students prefer printed materials over online materials (e.g., McGowan et al., 2009; Shepperd et al., 2008; Terpend et al., 2014; Vernon, 2006); however, our findings differ. It is not clear why this may be the case. One possibility is the wording of our questions. Another possibility is students' prior experiences with each type of textbook. Both possibilities are further discussed in the Limitations section.

When asked to make a choice about their preference between an open and a traditional paper textbook, students in both conditions selected the text they were using. Students' preference for whichever text they had, online or paper, may be explained by behavioral economics, a tendency for people to like what they already possess (Ariely, 2008). Despite the admitted advantages of the free, open online text, students who had purchased the paper text experience ownership and consequently, value in what they possessed. Ariely (2008) refers to this as the “Ikea effect.” Thus, actual value and perceived value can be quite different.

In terms of advantages, students from both groups perceived ease of reading, convenience, ability to highlight and take notes, ability to quickly find a topic, and ability to keep as a reference as the biggest advantages of traditional textbooks, and cost, weight, and convenience as the biggest advantages of an open text. These findings are similar to previous research on open online resources as compared to traditional print ones (e.g., Hilton, 2016).

Limitations

There were several limitations in the current study. First, the study was quasi-experimental, so it is not possible to draw causal conclusions; however, both course sections were taught by the same instructor, using the same instructional strategies, in the same room, at the same time of day, using identical in-class materials, syllabi and sequencing, on alternating days of the week, which minimized the effect that these factors may have had on the dependent measures. Next, our sample size was limited (N = 60), which affected the power of our statistical analyses, and results only included one semester, which limits the scope of the findings. We also did not collect demographic data on race or ethnicity, nor did we collect prior test scores (e.g., ACT, SAT), which could have served as covariates in our statistical analyses. We recommend future studies collect this information if possible.

Question wording was also an issue. The wording on the survey did not include the terms “open textbook,” and instead emphasized that the text was “free” and “online.” This may have affected the way students responded to the questions. Further, Question 3 read, “I like the idea of buying a free textbook.” This likely caused confusion since a free textbook cannot be bought.

In terms of our measures, we modified questions that had been used previously in research studies examining student perceptions and use of textbooks (Gurung & Landrum, 2012; Lindshield & Adhikari, 2013; McGowan et al., 2009), including the Textbook Assessment and Usage Scale (TAUS; Gurung & Martin, 2011). Our survey questions did not undergo control for validity and reliability, so it is possible that our measure was lacking in these areas. Future studies will need to carefully modify the language used in each question and use questions for which validity and reliability standards have been met.

Additionally, our comparison of open online text versus a traditional paper text could indicate that rather than comparing open versus traditional textbooks, it was more of a comparison between online and paper textbooks. Prior research shows that students respond differently to online compared to paper texts (e.g., McGowan et al., 2009; Shepperd et al., 2008; Terpend et al., 2014; Vernon, 2006). Furthermore, students in the paper text condition may not have any experience with open or online texts, which could have skewed their perceptions about these texts. In future studies, researchers should ask students about their prior experience with open or online textbooks.

Finally, although we randomly assigned conditions to each class, we did not measure students' prior knowledge about psychology. Therefore, we do not know if one class had more existing knowledge than the other about the topic. This is a potentially large flaw in the design of the study; however, it should be noted that this course was an introductory level psychology course, and most students were taking it for the first time. If any students had prior knowledge of the subject before taking the course, we believe this would have a minimal effect on our findings. In the future, researchers should measure pre-existing knowledge through a pre-test in order to eliminate the possibility of this knowledge having an effect on group differences.

Conclusions

Prior research on open psychology textbooks have offered valuable insights into the effect these resources have on student performance, perceptions, and textbook use. It is important that this research continue, while more carefully minimizing possible confounds through more closely controlled research studies (Griggs & Jackson, 2017). Open textbooks and other OER have the potential to make learning more accessible to students by providing a free, downloadable alternative to traditional publishers' texts in a variety of disciplinary areas, including psychology. Our findings, although with limitations, have implications for students, educators, and administrators. By using OER, students can not only save on textbook costs, but also have less restricted access to course materials; educators can use the materials in ways that best benefit their own course needs and teaching practices; and because of the decreased financial burden students face, administrators in higher education may find OER appealing because of the possible increase in retention, progression, and graduation rates. At this point, the body of empirical evidence is growing, and we call on others to continue addressing these questions through their own research. By doing so, the body of literature on OER will continue to grow and improve, helping to further inform best teaching practices.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Judy Orton Grissett, Ph.D. is an associate professor of Psychology in the Department of Psychology and Sociology at Georgia Southwestern State University in Americus, Georgia. Dr. Grissett teaches several psychology courses, including research methods and introductory psychology. Her research interests are in educational psychology.

Charles Huffman, Ph.D. is a professor of Psychology and Department Chair in the Department of Psychology and Sociology at Georgia Southwestern State University in Americus, Georgia. Dr. Huffman teaches several psychology courses, including history and systems of psychology and cognitive psychology. His research interests are in cognitive psychology.

References

Allen, G., Gruman-Alvarez, A., Molinaro, M., & Larsen, D. S. (2015). Assessing the impact and efficacy of the open-access ChemWiki textbook project. Educause: Learning Initiative. Retrieved from http://www.educause.edu/research-and-publications.

Ariely

(2008) Predictably irrational: The hidden forces that shape our decisions, New York, NY: Harper.

Bliss

T. J.

Hilton

III Wiley

Thanos

(2013a) The cost and quality of open textbooks: Perceptions of community college faculty and students. First Monday 18(1)): . doi: http://dx.doi.org/10.5210/fm.v18i1.3972.

Bliss

T. J.

Robinson

T. J.

Hilton

III Wiley

D. A.

(2013b) An OER COUP: College teacher and student perceptions of open educational resources. Journal of Interactive Media in Education. 2013(1), p.Art. 4. doi: http://doi.org/10.5334/2013-04.

Bowen

W. G.

Chingos

M. M.

Lack

K. A.

Nygren

T. I.

(2012) Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management 33(1): 94–111.

Bureau of Labor Statistics, U.S. Department of Labor. (2016, August 30). The economics daily: College tuition and fees increase 63 percent since January 2006. Retrieved from https://www.bls.gov/opub/ted/2016/college-tuition-and-fees-increase-63-percent-since-january-2006.htm.

Chiorescu

(2017) Exploring open educational resources for college algebra. The International Review of Research in Open and Distributed Learning 18(4))

Clinton, V. (2018). Savings without sacrifices: A case study of open-source textbook adoption. Open Learning: The Journal of Open, Distance, and e-Learning, 33(3), 177–189.

Delimont

Turtle

E. C.

Bennett

Adhikari

Lindshield

B. L.

(2016) University students and faculty have positive perceptions of open/alternative resources and their utilization in a textbook replacement initiative. Research in Learning Technology 24.

10.

Feldstein

Martin

Hudson

Warren

Hilton III

Wiley

(2012) Open textbooks and increased student access and outcomes. European Journal of Open, Distance and e-Learning 2: 1–9.

11.

Fischer

Hilton III

Robinson

T. J.

Wiley

(2015) A multi-institutional study of the impact of open textbook adoption on the learning outcomes of post-secondary students. Journal of Computing in Higher Education 27(3): 159–172. doi: 10.1007/s12528-015-9101-x.

12.

Gil

Candelas

Jara

Garcia

Torres

(2013) Web-based OERs in computer networks. International Journal of Engineering Education 29(6): 1537–1550.

13.

Grewe

Davis

W. P.

(2017) The impact of enrollment in an OER course on student learning outcomes. The International Review of Research in Open and Distributed Learning 18(4)

14.

Griggs

R. A.

Jackson

S. L.

(2017) Studying open versus traditional textbook effects on students' course performance: Confounds abound. Teaching of Psychology 44(4): 306–312.

15.

Gurung

Landrum

R. E.

(2012) Comparing student perceptions of textbooks: Does liking Influence learning?. International Journal of Teaching and Learning in Higher Education 2(24): 144–150.

16.

Gurung

R. A. R.

Martin

(2011) Predicting textbook reading: The textbook assessment and usage scale. Teaching of Psychology 38(1): 22–28. doi: 10.1177/0098628310390913.

17.

Hendricks

Reinsberg

S. A.

Rieger

G. W.

(2017) The adoption of an open textbook in a large physics course: An analysis of cost, outcomes, use, and perceptions. The International Review of Research in Open and Distributed Learning 18(4)

18.

Hilton

III (2016) Open educational resources and college textbook choices: A review of research on efficacy and perceptions. Educational Technology Research and Development 64(4): 573–590.

19.

Hilton

III Gaudet

Clark

Robinson

Wiley

(2013) The adoption of open education resources by one community college math department. The International Review in Research of Open and Distributed Learning 14(4): Retrieved from http://www.irrodl.org/index.php/irrodl/index.

20.

Hilton

III Laman

(2012) One college's use of an open psychology textbook. Open Learning: The Journal of Open, Distance and e-Learning 27(3): 265–272. doi: 10.1080/02680513.2012.716657.

21.

Illowsky

B. S.

Hilton

III Whiting

Ackerman

J. D.

(2016) Examining student perception of an open statistics book. Open Praxis 8(3): 265–276.

22.

Lindshield

B. L.

Adhikari

(2013) Online and campus college students like using an open educational resources instead of a traditional textbook. Journal of Online Learning and Teaching 9(1): ) Retrieved from http://jolt.merlot.org/index.html.

23.

Ma, J., Baum, S., Pender, M., & Bell, D. (2015). Trends in college pricing. The College Board. Retrieved from http://www.collegeboard.org.

24.

McGowan

M. K.

Stephens

P. R.

West

(2009) Student perceptions of electronic textbooks. Issues in Information Systems 10(2): 459–465.

25.

Myers

D. G.

DeWall

C. N.

(2014) Psychology in everyday life. (3rd ed.), New York, NY: Worth Publishers.

26.

Ozdemir

Hendricks

(2017) Instructor and student experiences with open textbooks, from the California open online library for education (Cool4Ed). Journal of Computing in Higher Education 33(3): 177–189.

27.

Pawlyshyn, N., Braddlee, Dr., Casper, L., & Miller, H. (2013, November 4). Adopting OER: A case study of cross-institutional collaboration and innovation. Educause Review. Retrieved from http://er.educause.edu/articles/2013/11/adopting-oer-a-case-study-of-crossinstitutional-collaboration-and-innovation.

28.

Rockinson-Szapkiw

A. J.

Courduff

Carter

Bennett

(2013) Electronic versus traditional print textbooks: A comparison study on the influence of university students' learning. Computers and Education 63: 259–266.

29.

Shepperd

J. A.

Grace

J. L.

Koch

E. J.

(2008) Evaluating the electronic textbook: Is it time to dispense with the paper text?. Teaching of Psychology 35(1): 2–5. doi: 10.1080/00986280701818532.

30.

Siebenbruner

(2011) Electronic versus traditional textbooks: A comparison of college textbook formats. Journal on Excellence in College Teaching 22(3): 75–92.

31.

Stangor, C. (2010). Introduction to psychology. Retrieved from http://open.lib.umn.edu/intropsyc/.

32.

Terpend

Gattiker

T. F.

Lowe

S. E.

(2014) Electronic textbooks: Antecedents of students' adoption and learning outcomes. Decision Sciences Journal of Innovative Education 12(2): 149–173.

33.

Vernon

R. F.

(2006) Paper or pixels? An inquiry into how students adapt to online textbooks. Journal of Social Work Education 42(2): 417–427.

34.

Wiley, D. (n.d.). Defining the “open” in open content and open educational resources. Retrieved from http://opencontent.org/definition//.

An Open versus Traditional Psychology Textbook: Student Performance,Perceptions,and Use

Abstract

Keywords

Literature Review

Efficacy of Open Materials

Student Perceptions of Open Materials

Student Use of Open Materials

Current Study

Method

Participants

Materials

Textbooks

Survey

Procedure

Textbook implementation

Student course performance

Student perceptions and use

Results

Student Performance

Student Perceptions of Textbooks

Textbook Use

Discussion

Limitations

Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

Author biographies

References