Abstract
Designing first-year experiences for first-time in college students (FTICs) is challenging and the Covid-19 pandemic created additional issues for university professionals striving to support them. In 2020–21, universities struggled with a decrease in FTIC enrollment and first-year term-to-term persistence. Even with a plan to collect academic and psychological well-being predictive data in the summer, student success professionals, faculty, and data analysts knew little about how the use of that data would best serve first-year students entering college during the pandemic. This article describes a unique interdisciplinary response that one Hispanic Serving Institution (HSI) engaged in for creating an intervention plan to optimize success for incoming FTICs in the fall of 2021. Following the introduction, the description of the task force shows the wide variety of team members needed for a well-designed program. Next, the phases of development and implementation describe the timeline of activities for building the program. Finally, the reflection and advice section offers lessons learned in the interdisciplinary effort and plans for future programs. … student success professionals, faculty, and data analysts knew little about how the use of that data would best serve first-year students entering college during the pandemic.
In December 2020, the HSI's president and provost engaged in listening sessions across campus to check in on faculty after the first full semester of online instruction. Reflecting on the challenges faced by FTICs and at the invitation of the president and provost, the Educational Leadership and Policy Studies (ELPS) faculty researched the skills high school seniors were missing because of a disrupted year caused by the pandemic. The provost realized the need for additional interdisciplinary contributions and charged the Associate Vice Provost of Student Success (AVPSS) to increase FTIC intervention plans. The AVPSS assembled a group including faculty, data analysts, student success, and student affairs practitioners for the task force. The charge was to structure a comprehensive menu of interventions and design a rigorous evaluation methodology for incoming students that provided them with a toolkit to assess and advance their academic learning, sense of belonging, and career readiness skills. The selection of these priorities was informed by faculty research and previous experience in program design and is further detailed in the curriculum section. The interventions were to commence in the summer and continue into their first year. This article describes a unique interdisciplinary response that one Hispanic Serving Institution (HSI) engaged in for creating an intervention plan to optimize success for incoming FTICs in the fall of 2021
Task Force Members
In February 2021, to ensure commitment, a contract to engage faculty, graduate students, and student success professionals was crafted between an academic department (ELPS) and the Division of Student Success (DSS). The contract highlighted resources, time commitments, and deliverables with deadlines. Discussion among leaders of the DSS, Student Affairs, ELPS, and the institutional research office resulted in a list of required skill sets for those participating in this interdisciplinary project. In addition to embodying equity-mindedness, other criteria are listed below.
Leading strategic planning Designing intervention plans that apply relevant student success research and learning theory (both K-12 and higher ed) Designing and implementing research methods to evaluate the efficacy of both the intervention and the planning process Linking results of intervention to institutional performance metrics Managing projects Skillfully communicating with data scientists and practitioners Applying scholarship/knowledge to FTIC program design and implementation Prioritizing implementation steps Managing capacity and resources for implementation success Skillfully communicating across disciplines Mining data, merging data, and cleaning data sets quickly and accurately Applying multiple methods for efficient data collection and analysis Each group not only provided specific skills, but they also filled in common gaps of the other groups
Each group not only provided specific skills, but they also filled in common gaps of the other groups. For example, the practitioners’ needs to immediately design a solution were balanced by faculty members’ need to ensure the research design matched the implementation design. Similarly, the data scientist's data analysis recommendations were weighed against the findings’ value for both program improvement design and informing future educational research and practice.
Phase One: Gaining Clarity
The first phase of planning included introductions, development of visioning and purpose statements, and a needs assessment of incoming FTICs. Meetings were held online via Microsoft Teams at weekly intervals and were recorded for any missing members. Minutes, recordings, handouts, and research were saved to the shared location. The full team included administrators and practitioners within different divisions, many of whom had not previously met. Introductions were critical, especially given a second year of working online with Zoom fatigue (Bailenson, 2021) as a concern. The Assistant Vice Provost for Retention and Completion (Dr. Hickman), served as team leader, introduced the team's assignment, and led the intervention implementation team. ELPS faculty members led both the assessment/research methodology team and the process management team; facilitating the meetings, minutes, and follow-ups, and building upon the introductions to engage in meaningful dialogue. The first phase of planning included introductions, development of visioning and purpose statements, and a needs assessment of incoming FTICs
Dr. Ludvik grounded the team's charge within the context of current research on cultural wealth, career readiness, and social–emotional wellness. Following the visioning discussion, the team collectively developed a purpose statement to focus the work. Structure an equity-driven summer experience for our incoming students where they are empowered to pre-assess their career readiness skills and engage in relevant toolkit selections designed to enhance their academic learning, sense of belonging, and intrapersonal skills that significantly contribute to their success at UTA and beyond. The needs assessment illuminated the vast information that FTICs need to absorb as they enter their college experiences
Phase Two: Design of Intervention and Evaluation
The needs assessment illuminated the vast information that FTICs need to absorb as they enter their college experiences. It was clear that the summer experience, named Success U, could not fill every gap, so Hickman directed the team to focus on empowering students to assess and build their career readiness/intrapersonal skills. Refocusing based on the purpose, allowed the team to eliminate 48 of the 50 categories of information for FTICs, leaving the priority for accessing campus resources and academic learning plans. The team developed 10 different modules which were later compressed to five modules to fit the five-week schedule. Topics included technology and motivation, learning strategies and growth mindset, time and tasks plus academic planning, self-advocacy and overcoming bad grades, and resources.
Throughout the spring, Hickman navigated shifting expectations for the summer experience as funding opportunities evolved and as administrators considered offering the experience to all FTICs versus select groups of FTICs. Robert adjusted the evaluation and research design and continued to monitor the process and track items that were missed. By mid-April, Hickman updated the team with funding and staffing plans to offer a choice of two sessions to all incoming FTICs. Hiring for the peer-led team learning model (PLTL; Gosser et al., 2001) began in late April, with most peer educators recruited from their previous peer mentoring experience at the university. The team focused the remaining meetings on reviewing activities completed and those needed for program implementation, curriculum development, and data collection.
With the tight timeline, the team's commitment to providing a research-based intervention prevented the temptation to devolve into scattered silos focused on last-minute tasks. Nonetheless, the practitioners were under extreme pressure to deliver a quality program within less than two months while ensuring that orientation continued along with other Student Affairs programs. Leads of each working group provided weekly updates and the team shifted priorities wherever necessary. Meeting logs indicate that many evening and weekend sessions were needed to meet the timeline. Regarding research design, the faculty and data scientists provided support and ensured that as plans were created, potential barriers were identified and mitigated. For example, for the preintervention survey, the team minimized the number of steps for student completion and for accessing the data by pretesting the software, method of distribution, and collection. An additional benefit of the interdisciplinary team was the ease of working across departments and thus, increasing capacity. With access to marketing, data, and curriculum support and the cooperation of other university divisions, the first round of program recruiting began in early May. The team developed 10 different modules which were later compressed to five modules to fit the five-week schedule An additional benefit of the interdisciplinary team was the ease of working across departments and thus, increasing capacity
Program Implementation Details
The program design included using the PLTL model designed by Gosser et al. (2001) in which peer educators (college or graduate students) were assigned groups of 10–15 students in which they would lead a weekly online meeting of the full group and meet individually at least once in the five-week session for guidance. Peer educators received a week of training in university guidelines for peer leadership and the predesigned program curriculum. A key component of PLTL is the leader meeting weekly with staff liaisons to discuss weekly facilitation of materials to ensure that peer educators are skilled facilitators (Ewing, Unite, Franco & Hickman, 2021). A Canvas course provided written materials and separate sections enabled easy communication between peer educators and their groups. Meetings occurred through Microsoft Teams.
Curriculum Details
The curriculum design was informed by Ludvik's research on intrapersonal competency cultivation and experience designing and researching FTIC student interventions within HSIs and their efficacy. Ludvik's research demonstrates the importance of students feeling a sense of belonging in order to engage in learning. The team also considered research demonstrating the needs of first-generation and at-risk students (e.g., van der Zanden, Denessen, Cillessen, & Meijer, 2019) for academic learning and career readiness skills. Hickman's research on PLTL and experience in implementing it, in addition to her experience leading professionals in multiple formats significantly informed curriculum design and implementation.
Data Collection
In the design phase, team members determined variables needed for program analysis and selected software and modalities based on both ease of use for students and easy data collection. Once students completed a Microsoft Form to enroll in Success U, an indicator was assigned through the student information system to allow automated registration in the Success U Canvas course. To measure the effectiveness of Success U, preintervention and postintervention survey data were collected in addition to open-ended responses within lesson discussions. Students were asked to complete a survey prior to the session containing portions of the Ryff Scale and the full Sense of Belonging and Metacognitive Awareness Inventory. The survey was issued using Question Pro, which allowed the later merging of student demographic information. The postsurvey was administered during the first two weeks of the fall FTIC course to ensure enough time between the pretest and the posttest. In the design phase, team members determined variables needed for program analysis and selected software and modalities based on both ease of use for students and easy data collection
Phase Three: Implementation
As the starting date for the program approached and the design was complete, team meetings focused on specific issues surrounding implementation. For example, several students mistakenly registered for both sessions, so it was not clear to select only one session. Most troubleshooting was completed by practitioners as the issues arose, so the team focused on documenting issues for future process improvement and then shifted into focusing on data collection. The team ended weekly meetings in mid-June and followed up as needed.
There were challenges to the implementation of Success U, though the program ran successfully and served 790 students. For example, the team hoped that all Success U Students would first attend the summer orientation program implemented by the Division of Student Affairs. However, the programs overlapped and thus for some students, they were not able to engage in the programming in the expected scaffolded design manner. Over 600 Success U students qualified for the program completion scholarship of $350, funded by Higher Education Emergency Relief funds (HEERF). There were challenges to the implementation of Success U, though the program ran successfully and served 790 students
Reflection and Advice
Creating and implementing a new intervention program is challenging within any context, and responding to unique student needs as a result of an extraordinary social experience with an expedited timeline was equally extraordinary. The first step toward improvement is to begin plans as early as possible, at least 10 months prior to initial implementation. As the first online program focused on identifying and building intrapersonal competencies, this was an opportunity to build skills that FTICs missed in their final year of high school. While it was overwhelming to implement the full list of FTIC needs in the context of the larger interdisciplinary group, the process conversations generated key understandings that would later be useful throughout subgroup implementations. For example, team members shared “I never realized that,” and “Oh, that explains so much,” while others had the opportunity to explain the consequences on first-term enrollment when an FTIC neglects to meet with their advisor in the summer. So, while the early visioning and discussion work slowed progress on specific intervention planning, it provided the interdisciplinary group members time to consider student success activities, the intentional scaffolding of FTIC information within the greater context of all FTIC needs, and time to align research and data resources together for optimal use. In essence, this first phase planning process gave faculty, student success and student affairs practitioners an opportunity to walk through and apply the American College Personnel Association's curricular approach (Kerr, Edwards, Tweedy, Lichterman & Knerr, 2020) to previously siloed activities.
The largest change planned for the following year was to compress the two session options into one session held later in the summer. This allows more time for recruitment, planning, and hiring of the PLTLs. Starting later in the summer will also leave more time for communication with FTICs, both at the June orientation sessions and through push notifications. Without HEERF funding, AVPRC plans to target intervention to one specific group of FTICs; informed by the previous year's analysis. Preliminary data analysis revealed curriculum opportunities, particularly among those identifying as female, African American, and first-generation. In addition, the length of the intrapersonal competency survey is under review for shortening to improve survey completion rates.
The careful process mapping and use of research-based materials will facilitate improvement efforts for future FTIC interventions and will inform a curricular approach to the pre-FTIC experience. Overall, the use of an interdisciplinary team for designing an intervention plan allowed for a thoughtful program design that aligned with university goals used research-based materials, and included data collection and analysis, all of which will enhance future efforts for student success. As a bonus, the team valued meeting colleagues across the university enjoyed the experience of working together across departments, and founded future partnerships for continued student success efforts. Preliminary data analysis revealed curriculum opportunities, particularly among those identifying as female, African American, and first-generation
Footnotes
We love feedback. Send letters to executive editor Z Nicolazzo (