Strengthening Online Psychology Students’ Sense of Belonging with Social Annotation: An Experimental Study

Abstract

Sense of belonging is associated with postsecondary student success outcomes and has largely been studied within face-to-face course contexts. The increasing demand for online courses after the COVID-19 pandemic necessitates identifying ways instructors can foster belonging in their online courses. This study experimentally tested the effect of using social annotation on students’ sense of belonging in asynchronous online courses. Students in two psychology courses (N = 44) were randomly assigned to complete annotation assignments individually (control) or in groups using a social annotation platform (treatment). Students completed a course-level sense of belonging measure at the end of their courses and indicated their perceptions of social annotation. Sense of belonging was significantly higher in the social annotation condition, particularly for the introductory-level psychology course, and students reported positive perceptions of social annotation. Course performance did not differ between annotation conditions, and sense of belonging was not associated with final grades. These findings document the viability of social annotation as an evidence-based tool for promoting a sense of belonging in online psychology courses.

Online learning in higher education unexpectedly skyrocketed with the onset of the COVID-19 pandemic. In fall 2019, only 36% of postsecondary students enrolled in online courses, but this figure rose to 75% in fall 2020 (National Center of Education Statistics, 2022). Even after campuses began reopening in fall 2021, students continued enrolling in online courses at a rate much higher than prepandemic years, with some institutions continuing to see a higher demand for online courses (Weissman, 2023). Students’ attitudes toward online learning have changed as well. More students report positive perceptions of online learning, a preference for online or hybrid course formats, and a preference for using more digital tools and materials in courses (Seaman & Johnson, 2021; Shankar et al., 2023).

The prioritization of student belonging also shifted with the onset of the COVID-19 pandemic, with more institutions investing in resources aimed at helping students feel valued, supported, and secure (Lu, 2023). At the same time, research on attending to belonging in the classroom has grown (Gopalan & Brady, 2020; Lingat et al., 2022; Peacock & Cowan, 2019). The rise in demand for online courses, coupled with the increasing focus on belonging in the classroom, underscores the importance of identifying ways to foster belonging in online learning environments. The current study reports on using social annotation as a tool for improving sense of belonging in students enrolled in asynchronous online psychology courses.

Social Annotation

Social annotation is the process of interacting, sharing information, and producing knowledge using a web-based tool (Lazzara & Clinton-Lisell, 2022; Kalir et al., 2020). Social annotation programs allow students to engage with course materials using features such as highlighting, commenting, and upvoting, while also collaborating with peers (Novak et al., 2012). Instructors can assign students to collaboratively annotate textbooks, journal articles or other readings, webpages, podcasts, or videos. Importantly, social annotation platforms are free and easily integrate with most course learning management systems (LMS; i.e., Blackboard or Canvas). A variety of social annotation programs currently exist, including Perusall and Hypothesis (Clapp et al., 2021).

Social annotation gives students a different online discussion experience than LMS discussion boards. In particular, online discussions using social annotation are typically more authentic and dynamic (Adams & Wilson, 2020; Brown & Croft, 2020), partly because students’ in-the-moment thinking is better captured through social annotation (Adams & Wilson, 2020). Indeed, some students preferred to use social annotation rather than LMS discussion boards (Lazzara & Clinton-Lisell, 2022). Additionally, because students can ask questions about course material in real time, instructors are better able to monitor students’ learning with social annotation (Adams & Wilson, 2020). Students in undergraduate psychology courses also reported feeling more motivated for social annotation assignments compared to other course assignments (e.g., quizzes; Clinton-Lisell, 2023) and were more comfortable asking questions via social annotation than LMS discussion boards (Lazzara & Clinton-Lisell, 2022). Taken together, these findings may explain why students perceive improvements in their learning and understanding with social annotation (Lazzara & Clinton-Lisell, 2022). More pertinent to the current study, social annotation increases the number of peer-to-peer interactions in an online course (Adams & Wilson, 2020; Eisenreich & Disney, 2024), which likely contributes to students feeling a greater sense of community in courses that use social annotation (Lazzara & Clinton-Lisell, 2022).

College Student Sense of Belonging

Feelings of social belonging and being part of a group are basic human needs. Before higher-order needs such as knowledge and proficiency can be satisfied, a student's need for belonging must first be satisfied (Maslow, 1962; Strayhorn, 2019). In the context of higher education, sense of belonging is defined as the “extent to which students feel personally…supported by others in the school social environment” (Goodenow, 1993, p. 80). Scholars suggest two key attributes influence the development of a learner's sense of belonging: feelings of connectedness to a group, class, or institution, and feelings of being valued and accepted (Tovar & Simon, 2010; Vaccaro et al., 2015). Key metrics of college student success are associated with a greater sense of belonging, including GPA, retention, persistence, and graduation (Tinto, 2017; Walton & Cohen, 2011). In addition, several indicators of student achievement and well-being are influenced by sense of belonging, including mental health, motivation, self-efficacy, and social connectedness (Arslan et al., 2022; Gopalan & Brady, 2020; Pedler et al., 2022).

Prior research on sense of belonging within the context of the classroom has predominantly focused on face-to-face courses, including research specific to psychology courses. The growing literature in this area shows that specific pedagogical practices like the inclusion of identity safety cues in syllabi and course materials (Howansky et al., 2022; Maimon et al., 2023) and team-based learning (Madson et al., 2024) lead to a greater sense of belonging in undergraduate psychology students. Additionally, having more peer connections significantly predicts course-level sense of belonging in introductory psychology students (Yust et al., 2021). Only one of these studies assessed learning outcomes and found that greater belonging predicted higher final grades in introductory psychology (Yust et al., 2021).

The lack of research on sense of belonging in online contexts is concerning given students in online courses report lower sense of belonging than students in face-to-face courses (DiGiacomo et al., 2023; Lingat et al., 2022). Further, online students are more likely to report feeling loneliness, isolation, and as though they are “second-class citizens” (Kaufmann & Vallade, 2022; O'Shea et al., 2015), which may inhibit their sense of belonging (Lingat et al., 2022). Attending to online students’ sense of belonging is critical given its association with retention and persistence in online programs (Hart, 2012; Shaikh & Asif, 2022; Thomas et al., 2014). Further, fostering a sense of belonging in online students is an equity issue, as it may lower attrition rates for students from marginalized backgrounds (Archard, 2014; Thacker et al., 2022).

Despite the importance of attending to belonging in online contexts, more research is needed (Peacock & Cowan, 2019). A small number of existing studies identified outcomes associated with a greater sense of belonging in online learners, including better academic performance (Edwards et al., 2023) and greater perceptions of course enjoyment and usefulness (Zengilowski et al., 2023). Further, students have reported several key factors associated with their sense of belonging in online courses, with engagement from instructors, opportunities for social interaction, and learner discussion being the most important (DiGiacomo et al., 2023; Peacock et al., 2020).

While these self-report findings contribute to our understanding of the conditions under which students feel they belong in online environments, no existing studies have empirically investigated pedagogical interventions that foster a sense of belonging in online courses. The current study sought to address this gap by examining whether social annotation meaningfully impacts sense of belonging. Since social annotation creates more opportunities for student-to-student and student-to-instructor interactions and engagement (Adams & Wilson, 2020; Eisenreich & Disney, 2024), it follows that students’ feelings of connectedness and acceptance (i.e., belonging) may be strengthened through using social annotation.

Community of Inquiry Framework

The Community of Inquiry (CoI) is a well-established theoretical framework for studying the process of learning in online environments (Garrison et al., 2000). The framework consists of three interdependent elements necessary for effective online learning: cognitive, instructor, and social presence (Arbaugh et al., 2008). Cognitive presence is the extent to which learners can construct knowledge and deepen their understanding. Cognitive presence is reflected through the use of instructor feedback, course resources, and individual and group reflection and discourse (Garrison et al., 2000; Garrison, 2017; Peacock & Cowan, 2019). Instructor presence refers to the extent to which instructors design, plan, and facilitate courses to help students meet learning objectives, as well as to deepen their learning and understanding (Garrison, 2017; Peacock & Cowan, 2019).

More pertinent to the current study, social presence reflects the extent to which learners can “project themselves socially and emotionally, as well as their ability to perceive other learners as ‘real people’.” (Boston et al., 2009, p. 68). Social presence focuses on creating a sense of community in online environments through meaningful opportunities for collaboration and communication. This, in turn, leads to students feeling connected, valued, and as though they matter (Garrison, 2017; Peacock & Cowan, 2019). According to the CoI framework, a sense of belonging is closely tied to social presence (Archard, 2014; Peacock & Cowan, 2019). Indeed, some measures of social presence (Arbaugh et al., 2008) include perceptions of belonging, further suggesting the two constructs are related. Given that social annotation encourages students to collaborate and communicate, it stands to reason that it can be used to build social presence, and thus, may impact students’ sense of belonging.

Method

The current study examined whether social annotation strengthens students’ sense of belonging in online psychology courses. A between-subjects experimental design was used, with students randomly assigned to either a social annotation condition (experimental group) or an individual annotation condition (control group). Students in the social annotation condition completed annotation assignments in groups on Perusall, while students in the individual annotation condition completed annotation assignments individually on their course LMS. All other elements of the courses were identical for students in both conditions. At the end of the course, students were invited to complete a survey including a measure of course-level sense of belonging. Students’ perceptions of social annotation were also assessed. Overall, the following research questions were tested:

Would students in the social annotation condition report a significantly higher course-level sense of belonging compared to the individual annotation condition?

Would students in the social annotation condition perform better in their courses than students in the individual annotation condition?

Would a higher course-level sense of belonging predict better academic performance?

What are students’ perceptions of using social annotation in online courses? Are themes from their responses reflective of social annotation fostering a sense of belonging?

Participants

The current study involved one section of an online undergraduate Introduction to Psychology course (PSYC 111) and one section of an online graduate Foundations of Educational Psychology course (EFR 501) at a midsized, Midwestern public university. Both courses ran for six weeks during the summer 2023 session. The first author taught the PSYC 111 course, and the second author taught the EFR 501 course. The first four weeks of EFR 501 also included discussion boards requiring students to post a response to a prompt and at least four comments on their peers’ posts.

All 66 students enrolled in both courses were eligible to participate, but only 44 completed the survey. Of these 44 participants, 7 (15.9%) identified as men and 37 (84.1%) identified as women. Students ranged in age from 18 to 56 years (M = 25.81 years, SD = 8.73 years). Twenty-six students (59.1%) were enrolled in PSYC 111 and most (54.5%) were first- or second-year students from a variety of academic programs. Eighteen students (40.9%) were enrolled in EFR 501 and were all graduate students from various academic programs.

Regarding racial and ethnic identity, most students (75%) in the sample were White. The remainder of the sample included 5 (11.4%) Black or African American students, 2 (4.5%) Asian American or Pacific Islander students, 2 (4.5%) biracial students, 1 (2.3%) Hispanic or Latinx student, and 1 (2.3%) student who preferred not to respond. Notably, racial and ethnic identities were not equivalently distributed across the two courses. Most (8) of the participants with minoritized racial/ethnic identities were enrolled in PSYC 111. Ten students (22.7%) reported first-generation status, and almost all students (93.2%) reported not having used the Perusall social annotation platform before the study.

Materials

Social Annotation Platform. Students assigned to the social annotation condition used the Perusall platform to complete their annotation assignments. Perusall randomly assigns students to annotation groups and instructors can choose the size of the groups. Students in the current study remained in the same groups for all annotation assignments. Perusall allows students to highlight specific excerpts from a text and add a question or comment in the margins. Students are encouraged to respond to questions and comments from other students in their annotation groups, and instructors are also able to respond to questions and comments. Perusall has an autograding algorithm that instructors can tailor to suit their assessment goals. Grading criteria include annotation timeliness and quality, number of annotations, distributing annotations evenly throughout the assignment, reading the entire assignment, time spent actively reading, and getting upvotes or responses from other students. See Figure 1 for an example of an annotated document in Perusall.

Figure 1.

Example of an annotated document in perusall.

Annotation Assignments. Students in both courses completed six annotation assignments (one per week). The PSYC 111 students annotated open-access textbook modules and the EFR 501 students annotated research reports relevant to their course content. Students in the individual annotation condition were instructed to write at least four annotations on PDF copies of the assigned course material for each assignment and uploaded their assignments to the LMS for grading by the instructor. Students in the social annotation condition were instructed to post at least four annotations on the assigned course material in Perusall. Annotations included questions, comments, or reflections students had about anything from their assigned readings. Course instructors and peers in students’ annotation groups had the option to reply to students’ questions or comments. Although replying to peers was encouraged, only a handful of students in PSYC 111 did so for each annotation assignment, instead choosing to focus on writing the minimum number of annotations needed. Students in EFR 501, however, were more engaged and responsive to their peers’ annotations. For both conditions, similar grading criteria were used to evaluate and score annotation quality. Annotation assignments were worth 30% of students’ final course grade. Copies of the assignment instructions for each condition are available on the Open Science Framework (OSF) project page (Kelly & Clinton-Lisell, 2024).

End-of-Course Survey. The survey used in the current study included items from the Brief Course Belonging Scale (BCBS; Lingat et al., 2022), which measures course-level sense of belonging in postsecondary students across different course modalities. On the BCBS, students indicated their level of agreement with 10 items (e.g., “I feel like my contributions during class activities matter to other students in this course,” “I feel like other students in this course encourage me to do well,” or “I feel included by other students in this course”) on a 5-point Likert-type scale ranging from Strongly Disagree to Strongly Agree. The BCBS is a newly developed scale that will benefit from further psychometric validation, but its development followed best practices and was guided by feedback from the target population and content and field experts. Internal consistency for the items was strong (α = .89).

At the end of the survey, students in the social annotation condition were asked to answer an open-ended question regarding what they found interesting or enjoyable about social annotation. The themes for this item were coded through an inductive content analysis by the first author. A research assistant further coded a subset of the responses, and interrater agreement between the authors’ coding and the research assistant's coding was good. A copy of the survey is available on the OSF project page (Kelly & Clinton-Lisell, 2024).

Procedure

Before data collection, the authors obtained approval from the institutional review board, and an exempt protocol was granted. At the beginning of their courses, students were randomly assigned to either the individual or social annotation conditions through their course LMS (Blackboard). Students then completed weekly annotation assignments on either the LMS (individual condition) or Perusall (social annotation condition). During the final week of their courses, students were invited to complete the survey for bonus points. An alternative assignment was available for bonus points for students who chose not to complete the survey.

Results

Before analysis, data were carefully inspected, coded, and cleaned. No outliers were detected, and the data met the normality and homogeneity of variance assumptions. The final cleaned, coded, and de-identified dataset is available on the OSF project page (Kelly & Clinton-Lisell, 2024).

Sense of Belonging by Course and Condition

There were no a priori hypotheses about differences in sense of belonging between the two courses, but Course was still included as an independent variable in the analyses. A two-way analysis of variance (ANOVA) compared students’ sense of belonging in their courses based on condition (individual or social) and course (PSYC 111 or EFR 501). The main effect of Condition was significant, F (1, 43) = 5.635, p = .022, η_p² = .123, as well as the main effect of Course, F (1, 43) = 13.315, p < .001, η_p² = .250. However, these main effects were qualified by a significant interaction between the effects of Condition and Course on sense of belonging, F (1, 43) = 7.641, p = .009, η_p² = .160.

Specifically, sense of belonging was significantly higher in the social annotation condition than the individual annotation condition for the PSYC 111 students (p < .001, η_p² = .288), but there were no differences in sense of belonging between annotation conditions for the EFR 501 students (p = .801, η_p² = .002). Further, sense of belonging was significantly higher in EFR 501 students compared to PSYC 111 students in the individual annotation condition (p < .001, η_p² = .332), but sense of belonging did not significantly differ between the EFR 501 and PSYC 111 students in the social annotation condition (p = .528, η_p² = .010). Descriptive statistics for both conditions and courses are included in Table 1. An interaction plot highlighting the effects of both independent variables on the dependent variable is shown in Figure 2.

Figure 2.

Differences in sense of belonging between annotation conditions and courses.

Table 1.

Course-Level Sense of Belonging Means and Standard Deviations by Condition and Course.

Condition	Course	Mean (SD)	N
Individual	PSYC 111	33.86 (5.17)	14
	EFR 501	44.13 (6.81)	8
	Total	37.59 (7.59)	22
Social	PSYC 111	42.08 (3.70)	12
	EFR 501	43.50 (5.34)	10
	Total	42.73 (4.46)	22
Total	PSYC 111	37.65 (6.12)	26
	EFR 501	43.78 (5.86)	18
	Total	40.16 (6.68)	44

Students’ Perceptions of Social Annotation

Students in the social annotation condition also responded to an open-ended question about what they found interesting or enjoyable about using social annotation in their courses. The most frequent themes centered on being able to see other students’ comments and perspectives (32% of responses) and reply to other students’ questions and comments (23% of responses). Some students reported appreciating that others responded to their own annotations, including both students (27% of responses) and instructors (7% of responses). Notably, one student mentioned the social annotation assignments being the “only connection to the other students,” given it was an online course. Frequencies for each theme are included in Table 2.

Table 2.

Themes for Interest and Enjoyment with Social Annotation.

Theme	Frequency (n = 22)
Seeing peers’ comments and perspectives	32%
Peers responding to own annotations	27%
Replying to peers’ questions and comments	23%
Instructors responding to own annotations	7%
Connection to other students in the course	4%

Course Performance

To determine whether sense of belonging was associated with students’ course performance, a Spearman rank-order correlation was conducted. On average, students performed well in their courses, leading to a skewed grade distribution (see Figure 3). Across all conditions, final grades (M = 94.16%, SD = 9.76) had a nonsignificant positive relationship with sense of belonging, r (43) = .108, p = .484, indicating that a greater sense of belonging was not significantly associated with better course performance. Additionally, a two-way ANOVA compared final grades based on course and condition. The main effect of Condition was nonsignificant, F (1, 43) = .212, p = .648, η_p² = .010, with students in the individual condition (M = 94.59, SD = 8.28) performing similarly to students in the annotation condition (M = 93.73, SD = 11.24). The main effect of Course was significant, F (1, 43) = 7.336, p = .010, η_p² = .155, with students in EFR 501 performing better (M = 98.67, SD = .68) than students in PSYC 111 (M = 90.04, SD = 11.78). The Condition × Course interaction was nonsignificant, F (1, 43) = .237, p = .629, η_p² = .002.

Figure 3.

Scatterplot for sense of belonging and final grades.

Discussion

Overall, the current study shows that social annotation can effectively strengthen course-level sense of belonging in students enrolled in online psychology courses. To our knowledge, this is the first study highlighting the utility of social annotation as a tool for improving online students’ sense of belonging. Given the shortage of empirically supported belonging interventions for online learners (Lingat et al., 2022), this is an important contribution to the existing literature. Further, as noted earlier, students in online courses have reported a lower sense of belonging than students in face-to-face courses (DiGiacomo et al., 2023; Lingat et al., 2022). Findings from the current study offer one course-level solution for addressing this disparity.

Sense of belonging was significantly higher for introductory-level psychology students in the social annotation condition, but this effect did not hold for students in the graduate-level course. Sense of belonging among this group was equivalent between individual and social annotation conditions, and already quite high compared to the introductory-level students. This is likely because the graduate students were part of a cohort taking the same courses, as well as the aforementioned discussion board assignments the students completed in their course.

Given that social annotation led to a greater sense of belonging for introductory-level online students, this speaks to the potential for social annotation (along with other tools that promote online interaction) to address common retention and persistence issues in this population of students. Specifically, students with less online course experience (Traver et al., 2014) and lower academic standing (Tello, 2008) are also less likely to persist in online courses. However, sense of belonging is a protective factor against student withdrawal from online courses (Thomas et al., 2014). Thus, online course instructors serving students in introductory-level psychology courses are encouraged to consider using social annotation as a way to foster belonging. Further, prioritizing students’ sense of belonging in online introductory-level psychology courses may influence the decision to take additional psychology courses, as well as serve as a proxy for the likelihood of their sense of belonging in the discipline (Yust et al., 2021).

Moreover, findings from the present study highlight the importance of student interaction in asynchronous online psychology courses (Delahunty et al., 2014). Social annotation affords students opportunities to engage with each other's questions and comments about course material. These interactions may foster belonging through greater feelings of connectedness to the course and feelings of being valued and accepted (Peacock & Cowan, 2019). In fact, students in the social annotation condition reported feeling more connected to others in the course. When viewed through the CoI framework (Arbaugh et al., 2008), these findings underscore the value of creating social presence in online courses, and how doing so can positively impact students’ sense of belonging.

Despite prior research finding an association between belonging and academic achievement (Abdollahi & Noltemeyer, 2018), greater sense of belonging did not predict better course performance in the current study. This finding is likely due to the overall high performance in the courses restricting the range to detect a prediction (see Figure 3), coupled with the small sample size. Previous studies finding positive correlations between sense of belonging and grades had much larger samples than the current study (Cwik & Singh, 2022; Edwards et al., 2021). Importantly, sense of belonging may be more critical for student retention in higher education programs rather than individual course performance (Ahn & Davis, 2023; Pedler et al., 2022), and the design of this study did not allow for continuation in degree progress to be examined. Social annotation also did not lead to higher final grades, likely because the annotation assignments were worth a small portion of students’ final grade in each course.

Limitations and Future Directions

The present study was limited by the small, predominantly female and White sample. Future research should seek to replicate these findings across different types of psychology courses, as well as to explore whether social annotation differentially impacts sense of belonging based on students’ racial and ethnic identities. It is possible students from underrepresented racial groups would particularly benefit from social annotation, given that student-to-student and student-to-instructor interaction supports belonging and engagement among Black or African American and Hispanic students in online STEM courses (Thacker et al., 2022). However, it is important to be mindful of potential harm that may happen in social annotations for students from minoritized backgrounds. For example, social annotations could become a space for expressing microaggressions or centering focus on dominant identities (Brown & Croft, 2020). Therefore, faculty must incorporate social annotation with an equity mindset and with careful monitoring of interactions in digital learning spaces (Brown & Croft, 2020).

Additionally, future research could compare social annotation to LMS discussion boards to determine whether one medium is more beneficial for students’ course-level sense of belonging. Past research found students were more active and reported more indicators of social presence when using alternatives to online discussion boards such as Voice Thread and video discussion (Guo et al., 2022; Xiu & Thompson, 2020). This same pattern of findings could emerge with social annotation since students find the discussions to be more authentic and dynamic than LMS discussion boards (Adams & Wilson, 2020; Brown & Croft, 2020). The combination of more frequent activity and perceptions of authenticity could translate to a greater sense of belonging in courses using social annotation compared to LMS discussion boards. This knowledge could be especially pertinent for instructors looking to find ways to foster belonging in their online courses.

Conclusion

This study found students’ course-level sense of belonging was strengthened by using social annotation for assignments in their asynchronous online psychology courses. Notably, the effects of social annotation on sense of belonging were still strong despite the shorter course duration. Altogether, this research adds to the growing literature on evidence-based ways to foster belonging in online course environments.