Sage Journals: Discover world-class research

Abstract

This study compares the academic performance and perceptions of 114 undergraduate students enrolled in an abnormal psychology course. Specifically, this study focuses on whether face-to-face (F2F) or blended modalities are associated with student learning outcomes. In this study, data analysis was based upon the examination of end-of-course grades, final exams, and an end-of-course survey. The data revealed that the same course presented in a F2F and a hybrid modality was associated with nearly identical learning outcomes in terms of student evaluations and final exam scores. However, students did note differences in course delivery in terms of time, assessment, and overall structure.

Keywords

Blended online hybrid perceptions learning

Introduction

A majority of degree-granting public institutions offer at least some form of distance education, and since 2003, the rate of online enrollments has exceeded the rate of enrollment in overall higher education (Allen & Seaman, 2015). In the fall of 2015, just over one in four (28%) students were enrolled in at least one distance course, and 2.8 million students were taking courses exclusively online (Allen, Seaman, Poulin, & Straut, 2016).

Distance education can include either online or blended courses. Allen and Seaman defined a blended course as a, “course that blends online and face-to-face delivery [where a] substantial proportion of the content is delivered online” with “a reduced number of face-to-face meetings” (Allen and Seaman, 2010, p. 5). The online component of the courses may include either synchronous or asynchronous learning resources (Hess et al., 2016); however, an effective blended learning course has a proper balance between online and face-to-face (F2F) learning (Jobst, 2016).

Several explanations for the rise in online and hybrid education have been offered including financial savings (Bowen, Chingos, Lack, & Nygren, 2014), student-driven integration of the Internet in their courses (Biddix, Chung, & Park, 2015), and flexibility and convenience for students and faculty (Lightner & Lightner-Laws, 2016; McWhorter, Delello, & Roberts, 2016). Whether these courses are equivalent to F2F counterparts in terms of engagement, motivation, and achievement remains in question (Collins, Weber, & Zambrano, 2014; Jaggars, 2011). Some students have also reported less positive perceptions of online learning (Johnson & Mejia, 2014), and online education may retain fewer students than F2F courses (Fetzner, 2013). Other studies, however, have shown blended instruction to be more effective than either F2F or Internet-based courses (United States Department of Education, 2010).

Overall, the literature presents mixed evidence regarding whether blended learning produces different student learning or satisfaction outcomes than F2F formats (Choi, 2013; Davies, Dean, & Ball, 2013). Thus, the purpose of this study is to compare the academic performance and perceptions of undergraduate students who completed a F2F abnormal psychology course to those who took the same course in a hybrid model taught by the same instructor.

Online Learning

Online classes are a form of distance learning available completely over the Internet with no F2F interaction between an instructor and students (Helms, 2014). Some of the benefits of online classes include a cost-effective price point, postsecondary credit equivalency, and the ability to provide education to anyone with Internet accessibility (Bartley & Golek, 2004; De la Varre, Keane, & Irvin, 2011; Gratton-Lavoie & Stanley, 2009).

Given the wide-variety of terms and course formats that may be used to refer to online education, synthesizing study results can be difficult, and research on student outcomes in online classes has been varied. Generally, online courses are not worse than F2F courses in student outcomes or student and faculty reports (Cook & Steinert, 2013; Yen & Abdous, 2011). For example, the systematic review by McCutcheon, Lohan, Traynor, and Martin (2015) found evidence that online learning resulted in at least as good clinical skills and knowledge in nursing students as traditional F2F education. Certain courses, particularly math and lab-based ones, may have unique barriers to online implementation such as coordinating professor-facilitated discussions or synchronizing feedback during assignments. Scherrer (2011), for example, compared student course averages between a F2F, hybrid, and two online undergraduate statistics courses and found that both sections of online students produced significantly lower averages than those in the other course formats. Additionally, students have rated laboratory-based social sciences, like psychology, to be more difficult to take online (Van Doorn & Van Doorn, 2014).

Students’ success in an online class may also be related to factors beyond course design. For example, McCutcheon et al. (2015) identified students’ age, computer experience, learning style, and attitudes toward technology as potential factors impacting the degree of engagement and satisfaction within an online course. Many online courses rely on a learning management system (LMS) to organize and distribute information to students, but students’ satisfaction with these systems may also be predictive of their satisfaction with the course (Marzilli, Delello, Marmion, McWhorter, & Roberts, 2015; Rubin, Fernandes, & Avgerinou, 2013).

Blended/Hybrid Learning

Helms (2014) described blended education as incorporating both online and F2F characteristics into a single course. This definition captures an important confound to comparing course administration formats because otherwise traditional F2F courses may also incorporate aspects of online curriculum. Blended learning may thus encompass F2F classes in which any course content is available online (e.g., recorded lectures or PowerPoints) as well as more traditionally blended courses. Helms recommended the use of “blended” over “hybrid” because these courses combine different but complementary approaches rather than layer opposing methods and formats. For the purposes of this study, we use the term “hybrid” to describe a particular type of blended course in which at least one weekly meeting of an otherwise F2F course is replaced with required online content, hybridizing a single course with F2F and online class meetings.

Blended learning can merge the relative strengths of F2F and online education within a flexible course delivery format. As such, this delivery form has a similar potential of online courses to reduce the cost of administration (Bowen et al., 2014) while addressing concerns of quality and achievement gaps that may come from online education. Advantages of blended courses include: convenience and efficiency for the student; promotion of active learning; more effective use of classroom space; and increased class time to spend on higher-level learning activities such as cooperative learning, working with case studies, and discussing big picture concepts and ideas (Ahmed, 2010; Al-Qahtani & Higgins, 2013; Lewis & Harrison, 2012).

Like online courses, a number of factors affect student outcomes and experiences in blended courses. Students in blended courses feel more accountability and experience a greater sense of community and personal attention than in a purely online course (Ahmed, 2010; Mansour & Mupinga, 2007). Opportunities for F2F interactions with the instructor and peers may be a critical component of successful hybrid approaches (Means, Toyama, Murphy, & Baki, 2013). Additionally, the incorporation of technology within blended classes is generally perceived positively by students (Tang & Byrne, 2007; Yudko, Hirokawa, & Chi, 2008). While López-Pérez, Pérez-López, and Rodríguez-Ariza (2011) found that blended education positively affected retention rates and exam grades, student grades still depended on the learning activities and students’ age, background, and attendance. Students’ social presence can significantly predict students’ delivery medium satisfaction (Arbaugh, 2014). Blended learning may outperform traditional classroom instruction in regard to student learning achievement outcomes; furthermore, computer support and the ability for students to interact with their peers, instructor, and/or course content enhance student achievement (Bernard, Borokhovski, Schmid, Tamim, & Abrami, 2014). Van Doorn and Van Doorn (2014) demonstrated that blended education may differ in its efficacy between traditional and non-traditional students but can be designed to meet the unique needs of both groups.

The flexibility of online and F2F education in blended courses presents a particular challenge to empirical studies because minor differences in the balance of online and F2F content could impact findings. For example, one study presented a highly flexible course design that let students select either an in-person, online, or interactive television course delivery for each scheduled class meeting, which resulted in similar outcomes and a high pass rate compared to F2F versions of the class (Lightner & Lightner-Law, 2016). In contrast, a randomized trial of a hybrid course on six different college campuses where students met F2F and had required online assignments once a week produced a modestly higher course completion rate among the hybrid courses (Bowen et al., 2014). Such differences in blended education design demonstrate the difficulty of aggregating results into a comprehensive set of recommendations; however, studies have generally found that various forms of blended education are comparable to, but neither better nor worse than, F2F equivalents (Bowen et al., 2014; Cook & Steinert, 2013).

Currently, the variability in terminology and course design in research on distance education precludes identifying an evidence-based, one-size-fits-all approach to using the Internet in education. For this reason, we suggest that studies focusing on courses within specific academic disciplines and settings be the focus of the literature in order to inform educational practice. Although there are examples in the literature of teaching human development and introductory psychology/social sciences in a hybrid format (e.g., Cottle & Glover, 2011; Jensen, 2011; Lewis & Harrison, 2012), we identified only one study in the literature that compared student outcomes when teaching abnormal psychology in F2F versus hybrid formats conducted with graduate students (Sullivan, 2016). Van Doorn and Van Doorn (2014) note that the learning, educational culture, and social needs of undergraduate and graduate students differ. Thus, additional research is needed to identify student perspectives on their experiences with F2F and hybrid instruction in an abnormal psychology course to determine relevant factors affecting student outcomes.

Methodology

This retrospective, non-experimental study implemented mixed-method data analysis techniques (Bryman, 2006) to explore archival course outcome data. This study uses student final exam grades, final course grades, and student survey responses from a standard, end-of-course survey given by the university; therefore, the two specific research questions utilized in this study were:

RQ1: How does a F2F class compare to a hybrid course as measured by grades?

RQ2: How does a F2F class compare to a hybrid course in terms of student perceptions?

Participants

The sample for this study included two classes of undergraduate students enrolled in an abnormal psychology course at a regional public university. This course was presented in the fall of 2014 in a F2F format with 65 (53 female; 12 male) students, and the hybrid course was taught in the spring of 2015 and included 49 students (39 female; 10 male).

Materials

In both classes, students were administered the same comprehensive final exam, which consisted of 82 matching and fill-in-the-blank questions. The final exam ranged from broad-based questions (such as identifying disorders by their diagnostic group), to more specific questions (such as listing symptoms of particular disorders), to detailed questions regarding treatment and etiology of the disorders. Additionally, each class was administered three unit exams. Each unit exam consisted of 80 multiple choice questions drawn from the text and lecture. Each test included material from the unit covered during that time period. In the F2F course, each exam was worth 25% of the final course grade. In the hybrid course, each exam was worth 20% of the final course grade.

Each chapter included in the course was accompanied by an optional, extra-credit quiz worth 5 percentage points toward the unit exam grade. Units 1 and 3 consisted of two chapters while Unit 2 had three chapters, so students could earn as much as 10 percentage points of extra-credit on the first and third unit exams or 15 percentage points for the second. In the F2F course, up to 10 percentage points of extra-credit were awarded on the final exam for completing clinical case studies. In the hybrid class, students were able to earn up to 8 percentage points on the final exam for a team review session.

In the hybrid format, the clinical case studies offered as extra-credit to the F2F class were a required part of the course structure. Students were asked to watch the clinical case studies online and use in-class time working on a group assignment related to the case study. These video assignments constituted 20% of the students’ final course grade in the hybrid class. Table 1 summarizes the various assignments and access medium for each of the courses.

Table 1.

Comparison of Content Delivery

Mode of Teaching	Face-to-Face (F2F)	Hybrid
Learning management system Blackboard	Online	Online
Lectures	F2F	Online
Group Work	F2F	F2F
Case Studies	Extra Credit-Online	Required-Online
Quizzes	F2F	Online
Exams	F2F	Online or F2F
Team Review	Not Offered	Extra Credit-F2F

The university where this study was completed assesses student satisfaction through a standard Likert-scale questionnaire that students may complete for each class in which they are enrolled at the end of every semester. Completing these questionnaires for their classes allows students to access their final grades for that semester early as an incentive for returning their feedback. The student evaluation instrument consisted of 13 Likert-scale questions and three open-ended questions about the course and the instructor (see Table 4). The three open-ended response items asked were, “What are some things that helped you learn in this course?”, “Did anything limit your learning in this course?”, and “Is there anything you wish the instructor might do differently the next time this course is taught?” Students’ participation in the course evaluation was voluntary and anonymous.

Course Context

To provide comparable learning experiences across the two classes, the content and structure of the two formats were designed to be as similar as possible. Both classes had F2F meetings scheduled for 8:00am. Students in both the F2F and hybrid classes were given access to Blackboard, a LMS. Both classes had approximately 34 lecture hours and approximately 6 ½ hours of mandatory exams and quizzes. The instructional design team at the university used its most advanced video technology to live capture every lecture given to the F2F course. This captured lecture series was used as the lecture content for the hybrid course, duplicating the experience as much as possible.

Though content was similar across both classes, the access medium for this content varied (see Table 1). Students in the hybrid course had the opportunity to choose whether to take the exams online or F2F. Both sections were offered quizzes that counted as extra-credit, but these quizzes were administered online for the hybrid section. Those students in the F2F class who completed the case study extra-credit did so online while the hybrid students completed their required quiz over the case study video online and performed their group work on the case in class.

Results

Quantitative Data Analysis

Final Examination Grade

In regards to RQ1, the final exam grade for the F2F and hybrid course were compared. The distribution of final exam scores was non-normal, W(114) = 0.89, p < 0.001, with both courses exhibiting a negative skew. To compare final exam performance across course formats, a Mann–Whitney U test was used. The median final exam score in the F2F (median (Mdn) = 81) and hybrid (Mdn = 83) course were not significantly different, U = 1,674, p = 0.64, Spearman’s r = 0.04. The grade distribution for the final exam for the F2F and hybrid course is shown in Table 2.

Table 2.

Course Format, Final Exam Grade, and Final Course Grade Summary Statistics

		Final Examination Grade			Final Course Grade
Course Format	N	Median	Mean (standard deviation)	Range	Median	Mean (standard deviation)	Range
Face-to-face	65	81	75 (24.7)	8–109	79.5	79 (14.3)	36–107
Hybrid	49	83	78 (21.0)	11–107	86	84 (11.0)	38–102

Note. All grades are reported in percentage points with extra-credit included.

Table 3.

Course Format and Extra Credit Earnings

		Chapter Quizzes		Final Exam Extra-Credit Activity
Course Format	N	Number who attempted (%)	Average points earned (standard deviation)	Number who attempted (%)	Average points earned (standard deviation)
Face-to-face	65	60 (92.3)	3.9 (2.0)	27 (41.5)	1.6 (0.5)
Hybrid	49	46 (93.9)	7.0 (1.8)	32 (65.3)	1.2 (0.6)

Note. Extra-credit is reported in percentage points.

Table 4.

Average Likert Scale Responses for End-of-Course Survey Results in both Abnormal Sections

Course	Average Instructor Rating ^a	Mann–Whitney U results
1. The instructor clearly defined the course objectives and expectations
Face-to-face	4.8 (0.5)	U = 842.5, p = 0.63, Speaman’s r = −0.05
Hybrid	4.8 (0.5)	U = 842.5, p = 0.63, Speaman’s r = −0.05
2. The instructor was prepared for each instructional activity
Face-to-face	4.8 (0.5)	U = 864.0, p = 0.83, Spearman’s r = −0.02
Hybrid	4.8 (0.5)	U = 864.0, p = 0.83, Spearman’s r = −0.02
3. The instructor communicated information effectively
Face-to-face	4.8 (0.4)	U = 816.0, p = 0.42, Spearman’s r = −0.09
Hybrid	4.7 (0.5)	U = 816.0, p = 0.42, Spearman’s r = −0.09
4. The instructor encouraged me to take an active role in my own learning
Face-to-face	4.7 (0.6)	U = 850.0, p = 0.73, Spearman’s r = −0.04
Hybrid	4.8 (0.4)	U = 850.0, p = 0.73, Spearman’s r = −0.04
5. The instructor was available to students either electronically or in person
Face-to-face	4.8 (0.5)	U = 843.0, p = 0.64, Spearman’s r = −0.05
Hybrid	4.7 (0.6)	U = 843.0, p = 0.64, Spearman’s r = −0.05
6. The course was organized well
Face-to-face	4.8 (0.5)	U = 806.5, p = 0.39, Spearman’s r = −0.09
Hybrid	4.6 (0.7)	U = 806.5, p = 0.39, Spearman’s r = −0.09
7. The instructor adhered to the expectations set forth in the syllabus and was clear about any changes that were made during the course
Face-to-face	4.8 (0.4)	U = 876.0, p = 0.96, Spearman’s r = −0.01
Hybrid	4.8 (0.4)	U = 876.0, p = 0.96, Spearman’s r = −0.01
8. The instructor was very interested in and enthusiastic about the subject matter
Face-to-face	4.8 (0.5)	U = 849.5, p = 0.66, Spearman’s r = −0.05
Hybrid	4.8 (0.4)	U = 849.5, p = 0.66, Spearman’s r = −0.05
9. The instructor used effective teaching methods that helped me learn
Face-to-face	4.6 (0.8)	U = 855.0, p = 0.79, Spearman’s r = −0.03
Hybrid	4.6 (0.6)	U = 855.0, p = 0.79, Spearman’s r = −0.03
10. Tests/assignments were returned promptly so I could track my progress in the course
Face-to-face	4.8 (0.5)	U = 867.0, p = 0.86, Spearman’s r = −0.02
Hybrid	4.8 (0.4)	U = 867.0, p = 0.86, Spearman’s r = −0.02
11. The instructor created an instructional environment in which I felt comfortable participating/expressing my opinion
Face-to-face	4.6 (0.8)	U = 850.5, p = 0.73, Spearman’s r = −0.04
Hybrid	4.7 (0.5)	U = 850.5, p = 0.73, Spearman’s r = −0.04
12. Overall, the instructor was (very unsatisfactory to excellent)
Face-to-face	4.7 (0.6)	U = 842.0, p = 0.64, Spearman’s r = −0.05
Hybrid	4.7 (0.6)	U = 842.0, p = 0.64, Spearman’s r = −0.05
13. Overall, this course was (very unsatisfactory to excellent)
Face-to-face	4.6 (0.7)	U = 860.5, p = 0.83, Spearman’s r = −0.02
Hybrid	4.6 (0.7)	U = 860.5, p = 0.83, Spearman’s r = −0.02

Note. For items 1 through 11, the scale was 1-Strongly Disagree, 2-Disagree, 3-Neutral, 4-Agree, and 5-Strongly Agree. For items 12 and 13, the scale was 1-Very Unsatisfactory, 2-Unsatisfactory, 3-Neutral, 4-Good, 5-Excellent.

Reports the mean response with the standard deviation in parentheses.

Final Course Grade

We also compared the final course grade across the F2F and hybrid courses to evaluate RQ1. Final course averages were not distributed normally, W(114) = 0.94, p < 0.001, with both sections’ final course grade distribution exhibiting negative skew. The Mann–Whitney U test was used to test whether the final course grades of the two sections differed. The final course grades in the hybrid section (Mdn = 86) were significantly higher than those in the F2F section (Mdn = 79.5), U = 2,014, p = 0.02, Spearman’s r = 0.23. The final course grade distribution for the F2F and hybrid course is shown in Table 2.

Due to differences in the number of assignments offered in each course, the final averages comparison was analyzed again to reduce the effect of those differences. First, we determined whether extra-credit differed between classes (see Table 3). Participation rates in the final exam extra-credit activity were significantly greater in the hybrid class than in the F2F class, χ²(1) = 6.32, p = 0.01, but did not differ for the unit exam extra-credit quizzes, χ²(1) = 0.11, p = 0.75. The hybrid course earned significantly more extra-credit on unit exams than the F2F class, U = 374.5, p < 0.001, Spearman’s r = 0.62; however, the F2F course earned more extra-credit toward their final examination than the hybrid course, U = 250, p < 0.01, Spearman’s r = 0.37. Based on these results, a new end-of-course average was calculated using the four exams for both classes without extra-credit and with equal weighting. The classes did not differ significantly on this rebalanced final average, U = 1,356, p = 0.24, Spearman’s r = 0.11.

Online Engagement

In order to better understand factors that may have affected performance in the hybrid class, we tested the relationship between student outcomes and the amount of time that students in the hybrid class spent in the LMS, which may be a proxy for the engagement of students in the online component of the class. On average, students in the hybrid class spent 28 hours (standard deviation = 15.3) in the LMS for the course. Length of time spent on the LMS was significantly and positively correlated with final course grade, Spearman’s r = 0.37, p < 0.01, but not with final exam grade, Spearman’s r = 0.13, p = 0.36.

Survey Item Responses

To evaluate RQ2, we examined the 13 Likert-scale questions and three open-ended questions on the end-of-course evaluation. Forty-four of 65 students provided evaluations of the F2F section (response rate = 68%), and 40 of 49 students provided evaluations of the hybrid section (response rate = 82%). The difference between these response rates trended toward significance in the two sections, χ²(1) = 2.80, p = 0.09.

Likert-Scale Questions

The average rating and standard deviation of both classes on the 5-point Likert-scale for each question of the student satisfaction survey is presented in Table 4. Mann–Whitney U tests were used to determine whether the classes differed significantly in their responses to these surveys, and these results are also presented in Table 4. None of these comparisons yielded a significant result.

Open-Ended Survey Questions

Responses to the open-ended survey questions were independently coded by two of the researchers. The researchers independently read and re-read the documents, identifying patterns and themes (Patton, 2014), and coded the data for each question through an inductive, comparative approach (Lincoln & Guba, 1985; Merriam & Tisdell, 2016). Then, the team members discussed the similarities and differences in the data—a form of analytical triangulation (Patton, 2014)—until consensus was reached. All authors reviewed and agreed with the coding results. Two themes and seven subthemes emerged from the data.

When students were asked on the end-of-course evaluation, “What are some things that helped you learn in this course,” the data indicated that instructor attributes were very important to them. Regardless of the format the course was presented in, participants from both the F2F and hybrid courses agreed that they valued the teaching style of the instructor. Students noted that the instructor had a high level of knowledge of the subject matter, was enthusiastic, and was personable (see Table 5).

Table 5.

Student Response Themes and Examples

Theme	Subtheme	Student Excerpt (Face-to Face)	Student Excerpt (Hybrid)
Instructor Attributes	Knowledgeable	“The teacher was very good at explaining every disorder and really knew every disorder and details on every disorder” “The lecture itself was very interesting with references to actual people with disorders” “[He] explains things well”	“[He is] very knowledgeable about the material” “[He] was the main reason I learned in this course … he took the time to explain and give examples of the different disorders we were covering.”
	Enthusiasm	“The instructor was very enthusiastic about the information and loved to help students” “The instructor is enthusiastic about the material”	“I enjoy his highly interactive teaching style and his enthusiasm” “He is very enthusiastic about the course”
	Personable	“[He] is very good at relating with students on a personal level which makes him more approachable” “He took the time to get to know his students” “The few times that I talked to him, he made me feel better”	“[He] is readily available via phone or email when you need” “He was very personable … and interested in his students” “… has a great passion for his students”
Course Delivery	Timing	“I didn’t like having a quiz at 8 am” “The time of the class made it hard for me to focus for the whole hour and twenty minutes”	“It was so flexible to the busy students and parents” “I feel that there’s so much to discuss in Abnormal Psychology that really requires more face-to-face time”
	Instruction	“He added PowerPoints online” “Read the book online” “Lectures were informative” “I feel like the large sums of material scared me away from really enjoying the reading”	“I feel I missed out on the lecture because the tests came from the textbook” “Having the lectures online in case there are any parts that we are not sure about is another great resource for backup”
	Assessment	“Add more assignments; I don’t like the only grades we receive come from tests” “I think he should give more quizzes”	“If I could [not] get a chance to take an exam at home, I still had a chance to take it in class” “Being able to take the test a couple of times”
	Course Structure	“Think about making the course a fully online class”	“This class is the model that every hybrid class should try to be” “Make it fully face-to-face”

Note. Themes for the open-ended questions were coded separately. This table combines all themes and subthemes irrespective of specific open-ended questions.

When students were asked, “Did anything limit your learning in this course,” 32% of the F2F and 40% of the hybrid course students wrote the word “no” or “nothing.” There were, however, some key differences found in the student perceptions of the theme of course delivery. The subthemes of timing, instruction, and assessment emerged from the data.

In regards to timing, some students in the F2F course noted a preference for a class that did not start at 8 o’clock in the morning. One student noted, “I’m just not a morning person.” Students in the hybrid course did not report any issues with the timing. In fact, several students reported that “the flexibility of the class and exams were wonderful” and that “I didn’t feel limited because [the hybrid course] was so flexible.”

In regards to course instruction students in both the F2F and hybrid course reported that the textbook was helpful to their learning; however, some students also remarked that there was too much material to learn. There were mixed reactions to the online course videos as several students in the hybrid course did not like the recorded lectures. For example, one student stated, “I struggled with this course particularly because he did not lecture; he recorded them and posted them online.” Another student, however, noted that they were helpful.

In regards to assessments, students in the F2F class did not like having so much of their grade based upon tests, wanted more multiple choice rather than short answer questions, and noted a preference towards more assignment-based activities.

When asked, “Is there anything you wish the instructor might do differently the next time this course is taught,” several of the students stated “no” (27% F2F/45% Hybrid). The main theme again, was related to how the course was implemented. Two subthemes prevalent in the data were that of assessment and course structure. Similar to the responses in the second open-ended question, most students in the F2F course wanted less testing and more assignments. For example, one student stated, “I feel like some portions of the tests aren’t really a true test of comprehension of the material.” Another noted, “Add some more assignments … I don’t like that the only grades we receive come from tests.” Another student in the hybrid course suggested that the professor, “find a way not to have a test the very first week of the semester.”

In regards to course structure, none of the students remarked that they would prefer a different course structure; however, there were mixed reactions. For example, one student in the F2F course noted that she lived far away from school and “just getting there on time is difficult.” Several students in the hybrid course reported that they would prefer this course be given as a F2F course only. Others reported that they loved the structure of the hybrid class. One student even suggested that the instructor, “Think about making the course a fully online class.”

Discussion and Implications

This study presented a unique opportunity to make a direct comparison between the same course presented in a F2F and a hybrid format in consecutive semesters by the same instructor. The quantitative data indicate that the courses were nearly identical in terms of student evaluations and final exam score. This was unexpected for a couple reasons: This was the first time the instructor had presented a course in a hybrid format, and we expected that he may experience problems that he could not anticipate and adequately respond to. These potential issues may have been minimized since the instructor had the advantage of previous experience teaching the course in both F2F and fully online formats. Second, we expected that the additional time investment of the hybrid format (through the required group case study) would negatively impact student interest and student course evaluations. This was not the case, though it should be noted that students in the hybrid format also lost less time in transport to and from class, probably making the overall time expenditure much closer than it initially appears. Bolstering this hypothesis is the fact that there were no negative comments regarding the extra time spent in the hybrid format over a typical F2F format even though a few students voiced a general preference for F2F learning.

In regard to students’ responses to the open-ended questions, we observed many more positive comments than negative regarding the hybrid format. When asked whether anything limited their learning or to identify ways to improve the course, only a few students cited specific features of the hybrid course while most reported no concerns or described other aspects of the course. Even when students expressed that the course should be offered in the F2F format, most did not identify any specific reasons for this preference or features of the hybrid format that were problematic.

Unlike other studies that have found hybrid courses to be no better or worse than F2F equivalent courses, our results demonstrate that the hybrid course had higher final grades than its F2F equivalent, though the effect size for this difference was small and final exam grades did not differ between the sections. This finding may be explained in a couple of ways. First, previous research has suggested that case studies can encourage critical thinking in students (Richardson & Ice, 2010). The hybrid format may have produced superior performance in that students learned to observe and analyze actual clinical case-studies, which is more difficult to implement in the F2F format due to time constraints. Incorporating the case study as a group project may have been a parsimonious way of improving students’ cognitive and social presence in the hybrid class—the latter of which may be a particular challenge for distance learning. Since less than half of the students in the F2F course (41.5%) completed the optional case study, it is plausible that at least part of the variance in student outcomes is due to the case study. Second, the differences in number of assignments and weighting of those assignments between the classes may have produced the observed differences. This possibility is supported by the null finding when the two courses were compared just on exam averages. Though a mathematically cleaner analysis, this rebalancing of the exam grades obscures important information related to the hybrid class’ performance on the extra-credit lecture quizzes: the hybrid course earned more extra-credit points on chapter quizzes than their F2F peers even though the two classes had equivalent participation rates, suggesting that the hybrid format had greater advantages regarding performance on the quizzes throughout the semester.

These findings suggest that the flexibility of the hybrid class allowed students to have more assignments, and thus more engagement with course content. The relative advantage of this flexibility allowed more engagement without affecting students’ overall learning, as reflected by the comprehensive final and overall average, or their experience in the course, as reflected by their end-of-course surveys. Evidence for the hybrid class having a more sustained engagement impacting course performance is the finding of a moderate correlation between time spent online and the final course average, but not the final exam, in the hybrid class. Additionally, the rate of participation for final exam extra-credit was greater in the hybrid than F2F class, though this may be due to the type of extra-credit activity offered to each class rather than due to true engagement differences.

Limitations

There are several limitations to the present study. One limitation of the study is that the decision to use the departmentally collected data for research was determined after its collection. As a result, the study lacks design controls or data collection that other studies may have included. While this absent data limit the explanatory power of the current study, there is an additional strength that we derive through a blind administration of the course: because there was not an intention to develop a study from the hybrid course, the class was taught solely with the goal of educating the students, reducing experimenter bias which is a notable issue in the current literature. For example, in Jackson, Jackson, and Chambers’ (2013) pilot study, the volunteering teachers attended two professional development sessions with one of the paper’s authors, introducing a risk of bias into the study. In the most methodologically rigorous study identified in our literature review, Bowen et al. (2014) noted that teachers volunteered to participate in the study and as such may be motivated to find significant results.

Another limitation to the generalizability of these results is the potentially unique experience that students in the undergraduate psychology program at this university may have with the LMS (Blackboard) and blended education. Most courses at the university offer asynchronous access to lecture content via Blackboard regardless of whether the course is classified as F2F, online, or hybrid. As a result, students at this particular university may have advantages in regard to familiarity with the LMS and blended education. Also, the experiences of students in our study may not be representative of other students who are less familiar with LMS software or integrating online components of a predominately F2F course with independent study (Rubin et al., 2013); however, Biddix et al.’s (2015) assertion that students’ use of personal technology in classes has already produced blended learning in otherwise F2F classes suggests that a clear division between F2F and blended courses may no longer be practical. Relatedly, students in the hybrid course had access to certain materials through the Internet where the F2F class did not have similar opportunity. The relative effect of Internet-based access to course content concerning student learning outcomes could not be directly assessed with the available data.

An additional limitation is the differential rate of engagement in the course surveys between the hybrid and F2F courses, which trended toward statistical significance. It is possible that those who did not complete these surveys represent a unique subset of opinions that were not otherwise represented. Because the university uses the same questionnaires for students across all courses, there were no questions related specifically to the class format. The results of the qualitative feedback may thus be limited in terms of obtaining students’ perceptions of the format in regards to meeting their expectations and/or facilitating their education. A related concern is self-selection bias among students enrolled in fall versus spring semesters. We lack data related to students’ age, enrollment status, or year in the program, which precludes our ability to determine whether the groups may differ on these important variables.

Conclusions

Our results suggest that the course format was associated with only a marginal difference on student learning outcomes or evaluation of an undergraduate abnormal psychology class. Based on our findings, a hybrid model where F2F time is spent on facilitated group case studies while lectures are presented online may be an effective way to leverage the relative strengths of the combined formats. Research in the future should continue to evaluate differences, or the lack thereof, between different formats of the same course in order to verify that certain types of courses, learning outcomes, and disciplines are amenable to adaptation with different forms of distance education. This case-by-case approach reifies the flexibility of blended education and the challenges of serving increasingly diverse student populations. When such research is combined with theoretical foundations, disparate course designs may be meaningfully compared. Utilizing this research would allow individual researchers to continue publishing easily accessible data and studies that can be collectively evaluated to identify the effect of various differences in course design and outcomes as the field moves toward a better understanding of the core components of effective instruction and delivery while still being aware of unique factors that may be tailored based on specific institutional needs.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

William F. Goette earned his M.S. in Clinical Psychology from the University of Texas at Tyler. He currently works as a psychometrist at the University of Texas Tyler Memory Assessment and Research Center. His research interests include cross-cultural neuropsychological assessment, acculturation psychology, and psychometrics.

Julie Delello, Ph.D., is an associate professor in the College of Education and Psychology and Director of The Center for teaching and Learning at The University of Texas at Tyler. Her research areas include new visual media technologies such as augmented and virtual reality applications, social media platforms, and the use of ePortfolios for learning.

Andrew L. Schmitt, Ph.D., is a licensed psychologist and a professor of Psychology at the University of Texas at Tyler. He teaches undergraduate and graduate courses in psychology and neuropsychology. Dr. Schmitt is the Director of the University of Texas Tyler Memory Assessment and Research Center and conducts research and clinical work in dementia and neuropsychological assessment.

Jeremy R. Sullivan, Ph.D., is a professor in the Department of Educational Psychology at the University of Texas at San Antonio. His areas of interest include psychoeducational assessment, psychometrics, and ethical, legal, and professional issues in school psychology.

Angelica Rangel earned her M.S. in Clinical Psychology from the University of Texas at Tyler. Her research interests include the neuropsychological assessment of cultural and linguistic minorities.

References

Ahmed

H. M. S.

(2010) Hybrid e-learning acceptance model: Learner perceptions. Decision Sciences Journal of Innovative Education 8(2): 313–346.

Allen

I. E.

Seaman

(2010) Class differences: Online education in the United States, 2010, Babson Park, MA: Babson Survey Research Group and Quahog Research GroupRetrieved from http://files.eric.ed.gov/fulltext/ED529952.pdf.

Allen

I. E.

Seaman

(2015) Grade level: Tracking online learning in the United States, Babson Park, MA: Babson Survey Research Group and Quahog Research GroupRetrieved from http://www.onlinelearningsurvey.com/reports/gradelevel.pdf.

Allen

I. E.

Seaman

Poulin

Straut

T. T.

(2016) Online report card tracking: Tracking online education in the United States, Babson Park, MA: Babson Survey Research Group and Quahog Research GroupRetrieved from http://onlinelearningsurvey.com/reports/onlinereportcard.pdf.

Al-Qahtani

A. A. Y.

Higgins

S. E.

(2013) Effects of traditional, blended and e-learning on students’ achievement in higher education. Journal of Computer Assisted Learning 29(3): 220–234.

Arbaugh

J. B.

(2014) System, scholar or students? Which most influences online MBA course effectiveness? Journal of Computer Assisted Learning 30(4): 349–362.

Bartley

S. J.

Golek

J. H.

(2004) Evaluating the cost effectiveness of online and face-to-face instruction. Educational Technology & Society 7(4): 167–175.

Bernard

R. M.

Borokhovski

Schmid

R. F.

Tamim

R. M.

Abrami

P. C.

(2014) A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education 26(1): 87–122.

Biddix

J. P.

Chung

C. J.

Park

H. W.

(2015) The hybrid shift: Evidencing a student-driven restructuring of the college classroom. Computers & Education 80: 162–175.

10.

Bowen

W. G.

Chingos

M. M.

Lack

K. A.

Nygren

T. I.

(2014) Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management 33(1): 94–111.

11.

Bryman

(2006) Integrating quantitative and qualitative research: How is it done? Qualitative Research 6(1): 97–113.

12.

Choi

E. M.

(2013) Applying inverted classroom to software engineering education. International Journal of e-Education, e-Business, e-Management and e-Learning 3(2): 121–125.

13.

Collins

Weber

Zambrano

(2014) Teaching business ethics online: Perspectives on course design, delivery, student engagement, and assessment. Journal of Business Ethics 125(3): 513–529.

14.

Cook

D. A.

Steinert

(2013) Online learning for faculty development: A review of the literature. Medical Teacher 35(11): 930–937.

15.

Cottle

N. R.

Glover

R. J.

(2011) Teaching human development: A case for blended learning. Teaching of Psychology 38(3): 205–208.

16.

Davies

R. S.

Dean

D. L.

Ball

(2013) Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Education Technology Research Development 61(4): 563–580.

17.

De la Varre

Keane

Irvin

M. J.

(2011) Enhancing online distance education in small rural US schools: A hybrid, learner-centered model. Journal of Asynchronous Learning Networks 15(4): 35–46.

18.

Fetzner

(2013) What do unsuccessful online students want us to know? Journal of Asynchronous Learning Networks 17(1): 13–27.

19.

Gratton-Lavoie

Stanley

(2009) Teaching and learning principles of microeconomics online: An empirical assessment. The Journal of Economic Education 40(1): 3–25.

20.

Helms

S. A.

(2014) Blended/hybrid courses: A review of the literature and recommendations for instructional designers and educators. Interactive Learning Environments 22(6): 804–810.

21.

Hess

Hagemeier

N. E.

Blackwelder

Rose

Ansari

Branham

(2016) Teaching communication skills to medical and pharmacy students through a blended learning course. American Journal of Pharmaceutical Education 80(4): 64, doi: 10.5688/ajpe80464.

22.

Jackson

L. C.

Jackson

A. C.

Chambers

(2013) Establishing an online community of inquiry at the Distance Education Centre, Victoria. Distance Education 34(3): 353–367.

23.

Jaggars

S. S.

(2011) Online learning: Does it help low-income and underprepared students? (CCRC Working Paper No. 26, Assessment of Evidence Series), New York, NY: Columbia University, Teachers College, Community College Research CenterRetrieved from http://ccrc.tc.columbia.edu/media/k2/attachments/online-learning-help-students.pdf.

24.

Jensen

S. A.

(2011) In-class versus online video lectures: Similar learning outcomes, but a preference for in-class. Teaching of Psychology 38(4): 298–302.

25.

Jobst

V. J.

(2016) Diving into the blended learning pool: One university's experience. Journal Of Higher Education Theory & Practice 16(4): 89–104.

26.

Johnson, H., & Mejia, M. C. (2014). Online learning and student outcomes in California’s community colleges. Public Policy Institute of California. Retrieved from http://www.ppic.org/content/pubs/report/R_514HJR.

27.

Lewis

J. S.

Harrison

M. A.

(2012) Online delivery as a course adjunct promotes active learning and student success. Teaching of Psychology 39(1): 72–76.

28.

Lightner

C. A.

Lightner-Laws

C. A.

(2016) A blended model: Simultaneously teaching a quantitative course traditionally, online, and remotely. Interactive Learning Environments 24(1): 224–238.

29.

Lincoln

Y. S.

Guba

E. G.

(1985) Naturalistic inquiry, Newbury Park, CA: SAGE Publications.

30.

López-Pérez

M. V.

Pérez-López

M. C.

Rodríguez-Ariza

(2011) Blended learning in higher education: Students’ perceptions and their relation to outcomes. Computers & Education 56(3): 818–826.

31.

Mansour

B. E.

Mupinga

D. M.

(2007) Students’ positive and negative experiences in hybrid and online classes. College Student Journal 41(1): 242–248.

32.

Marzilli

Delello

J. A.

Marmion

McWhorter

R. R.

Roberts

(2015) Exploring the perceptions of college students on the use of technology: What do they really think? International Journal of Sciences: Basic and Applied Research 24(2): 434–456.

33.

McCutcheon

Lohan

Traynor

Martin

(2015) A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. Journal of Advanced Nursing 71(2): 255–270.

34.

McWhorter

R. R

Delello

J. A.

Roberts

(2016) Giving back: Exploring service-learning in an online learning environment. Journal of Online Interactive Learning 14(2): 80–99.

35.

Means

Toyama

Murphy

Baki

(2013) The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record 115(3): 1–47.

36.

Merriam

S. B.

Tisdell

E. J.

(2016) Qualitative research: A guide to design and implementation, 4th edition. San Francisco, CA: John Wiley & Sons.

37.

Patton

M. Q.

(2014) Qualitative research and evaluation methods, 4th editions. Thousand Oaks, CA: SAGE Publications.

38.

Richardson

J. C.

Ice

(2010) Investigating students’ level of critical thinking across instructional strategies in online discussions. The Internet and Higher Education 13(1–2): 52–59.

39.

Rubin

Fernandes

Avgerinou

M. D.

(2013) The effects of technology on the Community of Inquiry and satisfaction with online courses. The Internet and Higher Education 17(1): 48–57.

40.

Scherrer

C. R.

(2011) Comparison of an introductory level undergraduate statistics course taught with traditional, hybrid, and online delivery methods. INFORMS Transactions on Education 11(3): 106–110.

41.

Sullivan, J. R. (2016). Teaching child and adolescent psychopathology: Traditional vs. hybrid formats. Manuscript submitted for publication.

42.

Tang

Byrne

(2007) Regular versus online versus blended: A qualitative description of the advantages of the electronic modes and a quantitative evaluation. International Journal on E-Learning 6(2): 257–266.

43.

United States Department of Education (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Retrieved from https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf.

44.

Van Doorn

J. R.

Van Doorn

J. D.

(2014) The quest for knowledge transfer efficacy: Blended teaching, online and in-class, with consideration of learning typologies for non-traditional and traditional students. Frontiers in Psychology 17(5): 324, doi: 10.3389/fpsyg.2014.00324.

45.

Yen

Abdous

(2011) A study of the predictive relationships between faculty engagement, learner satisfaction and outcomes in multiple learning delivery modes. International Journal of Distance Education Technologies 9(4): 57–70. doi:10.4018/jdet.2011100105.

46.

Yudko

Hirokawa

Chi

(2008) Attitudes, beliefs, and attendance in a hybrid course. Computers & Education 50(4): 1217–1227.

Comparing Delivery Approaches to Teaching Abnormal Psychology: Investigating Student Perceptions and Learning Outcomes

Abstract

Keywords

Introduction

Online Learning

Blended/Hybrid Learning

Methodology

Participants

Materials

Course Context

Results

Quantitative Data Analysis

Final Examination Grade

Final Course Grade

Online Engagement

Survey Item Responses

Likert-Scale Questions

Open-Ended Survey Questions

Discussion and Implications

Limitations

Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

Author biographies

References