Institutional Profiling for Educational Development: Identifying Which Conditions for Student Success to Address in a Given Educational Setting—A Case Study

Abstract

Thanks to a wealth of research on college student retention, we have today a good understanding of the factors that facilitate student success. However, actions taken to promote student success are far from always effective. The teaching–learning environments differ between institutions and provide, to a greater or lesser extent, the conditions that contribute to student success. The purpose of this case study is to test a method, based on a questionnaire and the use of effect sizes, to establish a “profile” of a particular university department to identify to what extent five conditions for student success (institutional commitment, expectations, support, feedback, and student involvement) contributed to student success in courses given at this department. The practical implications of the profiling method used in the study are discussed.

Despite insights from a wealth of research in higher education pedagogy, academic institutions still struggle to implement effective actions to promote college student success. The article addresses this issue by proposing, and testing, a method, based on a questionnaire and the use of effect sizes, to assess the extent to which a university department contributes to the success of its students. This method could be used by any department that would wish to embark on educational development projects to improve student success. Before starting pedagogical development projects, which are often costly and time-consuming, it is advisable to take stock of the situation and to try to assess the strengths and weaknesses of the department in terms of student success. The proposed method is based on Tinto and Pusser's (2006) model of institutional action for student success, which identifies five conditions for student success: institutional commitment, expectations, support, feedback, and student involvement. The method aims to establish the “profile” of a department and to identify the conditions that this department should focus on in order to promote its students’ academic success. More specifically, the study investigates the following research questions: (1) To what extent are the different conditions for student success identified by Tinto and Pusser (2006; see also Tinto, 2012) related to students’ academic performance (measured as their ability to complete the course in which they are enrolled) in the department investigated in this study? (2) To what extent does the strength of this relationship vary across the different conditions for student success? The practical implications of this “profiling method” is discussed as well as how this method could be used in other academic settings.

The phenomena of student success and retention in higher education, and their opposites (academic failure and student attrition), have been studied extensively for more than five decades (Aljohani, 2016; Burke, 2019; Tinto, 2006). The first theories of student attrition emerged in the 1970s. During this period, Vincent Tinto laid the foundation for his interactionalist model of student attrition, the student integration model (Tinto, 1975) and its further developments (Tinto, 1987, 2012, 2017; Tinto & Pusser, 2006). The model focused on the interaction between students’ individual characteristics and the academic environment. Although the student integration model came to occupy an “almost paradigmatic” position in research on student attrition and academic persistence (Barbera et al., 2020; see also Braxton, 1999; Nicoletti, 2019), other theories have been put forward. Schmitz et al. (2010) distinguish between two main categories: educational models on the one hand, such as Tinto's model, and motivational models on the other. The latter type of models looks at students’ individual characteristics, such as motivation or sense of self-efficacy. Models with financial and organizational starting points have also been developed (Bager-Elsborg et al., 2019). In addition to theories, many empirical studies have been conducted, using both quantitative and qualitative approaches. Overall, the research shows that student success and retention are complex, and to some extent unpredictable, phenomena and that the factors explaining these phenomena are multiple and intertwined (Agnidakis & Sundberg, 2018; Bager-Elsborg et al., 2019).

Despite the complexity of the factors contributing to student success, we have today good knowledge of the conditions for effective learning in higher education (Bolander Laksov et al., 2014) and for student success (Tinto, 2012). Various suggestions based on the broad evidence base available have been made in an attempt to move from theory to action and to turn what we know from theories and empirical studies into recommendations to try to enhance student retention and student success (see, for example, Braxton & McClendon, 2001; Braxton & Mundy, 2001; Tinto, 2012; Tinto & Pusser, 2006). However, although many resources have been invested in programs to enhance student success, the implemented actions have not always borne fruit. One reason for the limited success of those programs may be that the actions have not been sufficiently coordinated to address the complexity of these phenomena (Tinto, 2012). In many cases, the important issues of the teaching and learning environment, and how it is perceived by students, have also been neglected (Agnidakis & Sundberg, 2018; Bager-Elsborg et al., 2019; Tinto, 2012). However, Tinto (2012) sees a solution by focusing actions to promote student success on the “classroom”, which is for many students the only point of contact with the academic environment, especially for those who do not live on campus or have a family or a job to attend to in addition to their studies. Furthermore, support programs should focus primarily on the “conditions for student success”, such as institutional commitment, expectations, support, feedback, and student involvement, which, according to Tinto, are “known” factors that promote student success.

The extent to which the conditions for student success are established may vary across different educational settings, as suggested by Willcoxson, Cotter, et al. (2011), who surveyed students studying business administration at different levels (from the first year to the third year of study) at six different universities. The study found that the student perception of the elements shaping the conditions for student success varied across years and across universities. These findings suggest that university departments provide these different conditions for student success to a greater or lesser extent, and therefore the actions to be taken to enhance student success should differ between different educational settings. This is why targeted actions aimed at promoting student success in the particular context of a department should first focus on analyzing the specifics of the department in question and the students who attend it. This is what I propose to do in this case study, which aims to identify the conditions for student success in the specific context of a Swedish language department and a particular type of course: preparatory language courses.

Student success has often been defined as the completion of a degree (Tinto & Pusser, 2006). However, the question of what constitutes student success is debated. For instance, Cunninghame and Pitman (2020) suggest that it is too simplistic to equate student success with course or degree completion: Participation in student life has many benefits even for those who drop out, such as the opportunity to build networks, make new friends and develop a “broader understanding of the world around them”. However, the scope of the present paper is student success from an educational development perspective in order to enhance students’ academic performance: For the department, student success and students’ academic performance are important variables, as they contribute to the “financial health and well-being of the institution” (Stephenson et al., 2020). In other words, an institutional perspective was adopted in this study. This explains why student success was defined here as students’ ability to complete their course and take all the credits for which they registered. The study focuses on course completion and not degree completion because the department being studied in this paper mostly offers freestanding courses. Only a small share of students overall attends a program.

Theoretical Framework

Tinto and Pusser's (2006) model of institutional action for student success constitutes the theoretical framework of this article. It is itself based on the student integration model (Tinto, 1987) and was chosen because of its focus on variables that institutions can influence, such as the academic environment or the support and feedback that students receive. It does not include external factors such as the support of family and friends, the fact of having a job or social activities outside the university, as other models do. Because Tinto and Pusser's model focuses on aspects over which institutions may have some control, it can be used for what is called “institutional profiling for educational development”, as in the title of this article, i.e., it can be used for diagnostic purposes by a department that would like to take targeted actions to promote student success.

The Student Integration Model

The earliest works on student attrition were mostly descriptive and non-theoretical (Grayson & Grayson, 2003). They often had a psychological orientation and were concerned with students’ individual characteristics and shortcomings (Aljohani, 2016). In other words, these studies showed a tendency to see student attrition as the result of the personal characteristics of students, i.e., to “blame the victim” (Tinto, 2012). However, with models such as Tinto's (1975, 1987) student integration model, the focus shifted towards the interaction between students and the academic environment. Depending on how well it works, this interaction can result in a more or less successful integration of students, which can lead to a strengthening of students’ inclination to continue and complete their education. Conversely, a weakened sense of belonging can lead to dropout. The integration described by Tinto's model has two main components: social integration, which refers, inter alia, to students’ relationships with their peers, and academic integration, which concerns students’ interactions with the institution and, for example, how well students adapt and live up to the demands and expectations of the academic environment.

The student integration model was criticized for not taking external factors into account to a greater extent (Grayson & Grayson, 2003). This led Bean (1980) and Bean and Metzner (1985) to formulate an alternative model, the student attrition model, which overlapped with Tinto's model in several aspects but also included aspects such as the financial situation of students or the influence of their friends outside the university. Cabrera et al. (1992) compared the student integration model and the student attrition model in an empirical study of more than 2,400 students entering freshman class at a large urban university in 1988. Bean's dropout model explained 44% of the variance in students’ persistence and 60% of the variance in students’ intent to persist. The corresponding figures for the student integration model were lower: 38% and 36%, respectively (figures to be compared with 34% of the variance for persistence in Chrysikos et al., 2017). According to Cabrera et al. (1992), the differences in how well the models explained the variance of the variables are due to the inclusion of external factors, for example, parental and friend support and students’ economic situation, as examined in Bean's model. However, the authors judged the student integration model to be more “robust” in that 70% of its hypotheses were confirmed compared to about 40% for the student attrition model. Cabrera et al. (1993) tried to integrate both models. This led to marginal improvement, as 45% of the variance in persistence was explained by this integrated model—along with 42% of the observed variance for the intent to persist.

According to Cabrera et al. (1992) and Cabrera et al. (1993), Tinto's and Bean and Metzner's models are the only truly comprehensive theories of college persistence. They are also among the most influential and cited models in the research field (Aljohani, 2016; Burke, 2019). These two models have also been tested in several studies (Cabrera et al., 1992, 1993; Chrysikos et al., 2017; Schmitz et al., 2010), demonstrating their relevance—even if the proportion of variance explained by the models is sometimes described as “modest” (Chrysikos et al., 2017). As mentioned earlier, the choice of Tinto's models as the theoretical framework for this study is justified by their focus on variables over which academic institutions have a large degree of control. This is even clearer in a later development of Tinto's model, presented in the next section.

Model of Institutional Action for Student Success

Tinto and Pusser (2006) note that although research on student attrition is extensive, much less attention has been paid to the development of models that describe support actions that effectively promote academic persistence and student success. The authors attempt to remedy this deficiency by proposing their model of institutional action for student success, which identifies five conditions for student success:

institutional commitment: Institutional commitment is a key ingredient in college student success. It requires institutions to invest resources and create incentives and rewards to promote student success.

expectations and demands placed on students by institutions: Institutions should communicate high expectations and create learning environments that encourage students to study more.

support: Three types of support contribute to student success: academic, social, and financial support. Academic support is especially important for the large number of students insufficiently prepared for university studies. Different forms of social support can provide a “safe haven” for students feeling, for different reasons, out of place in the academic community.

feedback: Frequent monitoring and feedback, which is not to be confused with testing, also promote students’ academic success. Feedback includes not only different forms of assessment, such as classroom assessment techniques (see, for instance, Cottell & Harwood, 1998), but also entry assessment of learning skills and early warning systems.

student involvement: The last key element in Tinto and Pusser's model is involvement, which has often been referred to in the literature as engagement (Tinto, 2012) or as academic and social integration. The stronger the students’ sense of involvement, the more likely they are to continue and complete their studies.

In their model, the authors give examples of types of institutional actions that shape the above-mentioned conditions for student success and promote students’ academic success. These different examples of actions and programs are presented in Figure 1. The figure compiles the examples of institutional actions mentioned in Tinto (2012) and Tinto and Pusser (2006). Institutional commitment is an overarching prerequisite for the enhancement of the conditions for student success, which explains its position at the top of the figure. The institutional actions presented under the other four conditions for student success (expectations, support, feedback, and involvement) are grouped into different categories (indicated in bold type). For instance, student involvement, in the lower right part of the figure, can be fostered, among other things, through the implementation of “pedagogies of engagement”, of which cooperative or collaborative learning, problem-based learning, and project-based learning are examples.

Figure 1.

Institutional actions for student success (Tinto, 2012; Tinto & Pusser, 2006).

Synthesis and Hypotheses

Tinto and Pusser's (2006) model of institutional action for student success formed the theoretical framework of the present study. It builds on Tinto's student integration model, which is a relatively comprehensive theory and has been tested in several studies. Tinto and Pusser's model was chosen because of its focus on variables that academic institutions can influence. In this study, student success was equated with course completion. The study investigates to what extent the different conditions for student success were related to students’ academic performance and how strong this relationship was.

Tinto and Pusser's conditions for student success are determinants of student success. It was therefore anticipated that at least some of the five conditions for student success described by the model would allow for a distinction to be drawn between the students who completed their language courses and took all their credits and those who did not (see Participants and Sampling Procedure below). In other words, the group of students who took all their credits in the course in which they were enrolled was expected to show higher mean scores on at least some of the conditions for student success than the group of students who did not complete the course. With respect to the effect sizes of the conditions for student success, it was hypothesized that certain conditions play a larger role in student success in the particular academic setting of the study, which would result in different effect sizes across the conditions for student success. These results would thus provide a “snapshot” or a “profile” of the teaching–learning environment provided by the department investigated in the present study. They could therefore serve as a basis for discussion with the teaching and administrative staff, and even students, on the type of actions that should be implemented to enhance student success in the department in question.

Method

Participants and Sampling Procedure

The participants were students enrolled in a preparatory course, or equivalent, in Romance languages (French, Italian, Portuguese, or Spanish) or classical languages (ancient Greek or Latin) at a language institution at a large Swedish university during the spring, summer, and fall semesters of 2020. Preparatory courses are courses at the beginner level and do not require any prior knowledge of the language studied. The students taking these courses make up a large proportion of the student body at the department under study. Furthermore, the preparatory courses have a lower performance indicator compared to higher level courses at the department. ¹ These two elements explain why the study focuses on these courses.

An electronic questionnaire was sent by email to all the students enrolled in one of the department's preparatory courses in 2020 (n = 1,003, 61% of whom were women; age: M = 32.1 years, ² SD = 12.5), including those who had withdrawn under the semester. A total of 261 students responded to the questionnaire. ³ The response rate was 26%. Participation in the study was voluntary. The study is therefore based on a convenience sample. The respondents were distributed across the studied languages as follows: French 36%, Italian 22%, Portuguese 5%, Spanish 5%, ancient Greek 5%, and Latin 26%, which broadly reflects the distribution of students enrolled in the department's preparatory courses at the time of the investigation (French 31%, Italian 25%, Portuguese 7%, Spanish 8%, ancient Greek 6%, Latin 23%). Fifty-four percent of the 261 students who responded to the questionnaire completed their course, compared to 42% of the total of the 1,003 students enrolled in preparatory courses in 2020. In other words, the students who completed their course showed a higher propensity to respond to the questionnaire than those who did not complete their course.

Questionnaire

The importance of student-centered learning and taking into account the student perspective in educational development has been highlighted in several studies (see, e.g., Agnidakis & Sundberg, 2018; Chickering & Gamson, 1999). This explains why a questionnaire on students’ experience of their studies was used in the present study. Students’ perception of the learning environment has been shown to be related to the quality of students’ learning and their approaches to learning (Parpala et al., 2013), as well as to academic achievement (Diseth, 2007; Lizzio et al., 2002).

The questionnaire in this study contained 58 variables, formulated as statements that respondents were asked to evaluate by indicating the degree to which they agreed with the statements on a 7-point Likert scale ranging from Disagree completely (1) to Agree completely (7). The respondents also had the option of answering Don't know (0). The questionnaire is based on Tinto and Pusser's (2006) model of institutional action for student success: The questionnaire scales correspond to the five conditions for student success highlighted by Tinto and Pusser.

The questionnaire is an adapted translation of a questionnaire from a previous study (Willcoxson, Cotter, et al., 2011), which was part of a large Australian project on student retention, attrition, learning and support interventions during undergraduate business studies: The Whole of University Experience (Willcoxson, Manning, et al., 2011). ⁴ The original questionnaire items were translated to Swedish by the group of directors of studies at the department where the study was conducted. Some adjustments were made to take into account the specificities of the educational setting investigated in this study (see the questionnaire items in Appendix). Some questions were partially reworded due to the fact that the present study focuses on the students’ experience of specific courses, and not on programs or university studies in general. For instance, the item “I have a clear reason for attending university” was reworded as “I had a clear reason for attending the course”, ⁵ i.e., “university” was replaced by “the course” in this item as well as in several other items referring to “university” or university studies. Moreover, two items were added to capture the role of the department in student involvement (“I felt welcome at the department” and “I felt involved in the activities of the department”). A few questions that were not considered relevant to preparatory language courses were removed. This is the case concerning the item “I feel that my academic writing skills are adequate for my university studies”: Students participating in preparatory language courses have a limited use of their academic writing skills, as the preparatory courses focus on students’ acquisition of basic foreign language skills (listening, speaking, reading, and writing) and on the study of basic grammar and vocabulary. Students develop their academic writing skills in higher level courses. Another item that was removed from the questionnaire is “I am satisfied by the work experience opportunities offered by the university”. The reason for removal was that the department does not offer internship opportunities to its language students. The questions about students’ health and financial problems were also eliminated, as students may be reluctant to answer them. In addition, the actions that a university department can take to support the health or solve the financial difficulties of its students are limited. Another difference concerned the response variable, which was students’ intent to leave in Willcoxson and colleagues’ study, while respondents in the present study were asked to describe their activity on the course after the end of the semester (see below).

A first version of the questionnaire was sent to a group of teachers from the department. Changes were made after their comments. A pilot questionnaire was sent to the students taking preparatory courses at the department in the fall semester of 2019. 128 respondents answered. No further changes were made after this pilot.

Regarding the commitment variable, the questionnaire items measure two aspects of commitment: commitment to the course (see, e.g., “I had a clear reason for attending the course”, “The course was interesting”) and commitment of time (e.g., “I found it hard to manage my time effectively”, “I thought it was difficult to balance my social life and course work”). These two aspects reflect institutional commitment: It is the departmental staff who create the “framework” that allows students to invest in their studies, for example, by offering interesting courses or by distributing the workload over the semester (e.g., by spacing examinations over the whole semester instead of just at the end of the semester). However, it is important to note that the extent to which the departmental leadership is committed to student retention is weakly supported by the commitment scale in the questionnaire.

Variables and Data Analysis

At the beginning of the questionnaire, respondents were asked to identify themselves with one of the following four categories, which reflected their activity in the courses they had taken: (1) I took all credits for the course (n = 140, 54% of the respondents); (2) I took some of the credits for the course (n = 35, 13%); (3) I attended at least one lesson, but took no credits (n = 63, 24%); and (4) I did not attend a single lesson and did not take any credits (n = 23, 9%). The respondents who identified themselves with Category 4 did not have the experience of taking the preparatory course in which they were enrolled. Therefore, they are not included in the data analyzed. The reason why students can take some but not all the credits for one course (see Category 2) is that courses consist of different modules with credits attached to them. For example, one of the preparatory courses included in this study comprises seven modules, each corresponding to 3 to 7.5 credits. A student may pass the exams and get credits for only some of the modules. In this case, the whole course is not passed. It can only be validated if the student passes all the exams for all the modules.

The statistical analysis started with the screening of the data. The data had a low rate of missing values (0.9%). The missing data per item ranged from 0% to 2.5%. The Don't know responses and the missing values were replaced by item means (Dörnyei & Taguchi, 2009). Twenty-five respondents were eliminated from the data because they had left all items within one or several scales unanswered or had chosen the Don't know option for all items within one or several scales. The fact that these 25 students did not answer certain questions may be due to the fact that they did not follow the course they were enrolled in long enough to be able to comment on certain aspects of the course. That is why they were eliminated from the study corpus. Ultimately, the analysis was conducted on the responses of 213 participants. The scoring of the negatively worded items was reversed.

Willcoxson, Cotter, et al. (2011) conducted a factor analysis to combine the variables of the questionnaire into five factors corresponding to Tinto and Pusser's (2006) conditions for student success. The present study uses the scales established by Willcoxson, Cotter, et al. (2011). Cronbach's alpha was calculated for the five variables. As shown in the last column of Table 1, the Cronbach's alpha of the five factors ranged from .73 to .87, suggesting that the factors have a good reliability: They are above the recommended threshold of .7 (Tabachnick & Fidell, 2013).

Table 1.

Description of the Study's Variables.

Variable	Description	Example of statement	Cronbach's α
Commitment	Measures students’ perception of institutional commitment, i.e., whether the institution has succeeded in creating an educational setting conducive to student success.	The course was interesting.	.73
Expectations	Measures students’ perception of the department's ability to communicate high expectations and to create learning environments that make students study more.	I enjoyed the intellectual challenge of the course.	.87
Support	Measures students’ perception of the support that the teachers and the administration gave the students.	The teaching staff was sensitive to individual student needs.	.85
Feedback	Measures students’ perception of the quality of the feedback that they received on their performance.	I received helpful feedback on assessment tasks.	.82
Involvement	Measures how well the students felt integrated (academically and socially) in the university and at the department.	I had difficulty adjusting to the style of teaching at the university.	.79

The last phase of the statistical analysis was carried out as follows. The means of the scores of two groups of students were compared: the group of students who completed the course (Category 1, n = 138 after the elimination of the above-mentioned 25 students from the dataset) and those who did not take all the credits of the course (Categories 2 and 3, n = 75). Since the data were not normally distributed, the mean scores of two groups were compared using the non-parametric Wilcoxon rank-sum test. The tests were performed with the statistical program R (The R Project for Statistical Computing, 2022). A two-sided hypothesis test was conducted. The effect size of the conditions for student success was measured using a r-family effect size. The z-value, from which the effect size is calculated, was computed using the Monte Carlo method (Larson-Hall, 2015).

Ethical Considerations

The Swedish Act on the Ethical Review of Research Involving Human Subjects (Lag om etikprövning av forskning som avser människor, 2003) requires an ethical review of research activities involving human participants when the information processed includes the “sensitive personal data” covered by Article 9.1 of the European Union General Data Protection Regulation, such as “racial or ethnic origin, political opinion, religion or beliefs, trade union membership, genetic or health status or sexual orientation” (General Data Protection Regulation, 2016). As the questionnaire did not contain any questions concerning sensitive personal data, no ethical review was required. Participants were informed about the purpose of the study in the introductory text of the questionnaire. They gave their consent to the processing of their responses. The questionnaire was anonymous.

Results

The mean values of the scores on the five conditions for student success were compared between the group of students who completed their course and the group of students who did not (see Table 2).

Table 2.

Descriptive Statistics and Results From the Wilcoxon Rank-Sum Test.

Variable	Completed course		Non-completed course		W	z	p	r
	M (SD)	Range	M (SD)	Range
Commitment	5.39 (0.77)	2.65–7	4.88 (0.82)	3.31–6.54	6,955.5	4.15	<.001	.28
Expectations	5.46 (0.72)	2.74–6.88	5.21 (0.88)	3–6.89	6,158	2.29	.022	.16
Support	5.13 (0.85)	1.64–7	4.86 (0.83)	2.46–7	6,348.5	2.73	.006	.19
Feedback	5.97 (1.38)	1.5–7	5.48 (1.48)	2–7	6,194.5	2.51	.012	.17
Involvement	5.64 (0.69)	2.44–7	5.41 (0.75)	3.19–6.75	6,118	2.20	.028	.15

All the mean values of the scores on the conditions for student success were statistically significantly higher in the completed course group than in the non-completed course group. According to Cohen's (1992) benchmark, ⁶ the variable commitment was associated with a medium effect size (see the last column of Table 2). The other conditions for student success were associated with small to medium effect sizes.

Discussion

The hypotheses of the study were confirmed. First, the group of students who completed the preparatory course in which they were enrolled exhibited higher mean scores on all the conditions for student success than the group of students who did not take all the credits. (The hypothesis was that it would be the case for at least some of the conditions for student success.) There was therefore a relationship between the different conditions for student success on the one hand and course completion on the other. Second, as hypothesized, the strength of the relationship between course completion and the conditions for student success varied depending on the conditions for student success considered, as shown by the effect sizes computed for each variable. The mean values for the conditions for student success were relatively high in both groups, as shown in Table 2. This can be interpreted in two ways. Either these high mean values may be due to the fact that the department where the students took the preparatory courses already has created a relatively good framework for student success, or the instrument used to measure the conditions for student success had a weaker ability to discriminate between the groups. To determine which of these hypotheses is more likely, it would be desirable to supplement this study with interviews of students enrolled in this course (see below).

Tinto and Pusser's (2006) model of institutional action for student success, on which the questionnaire of this study is based, draws on several decades of research in higher education pedagogy. Not surprisingly, the results of the study were in line with previous studies. For example, Chickering and Gamson (1987, 1999) put forward seven principles for good practice in higher education. ⁷ Two of these principles directly intersect with Tinto and Pusser's (2006) conditions for student success: giving prompt feedback and communicating high expectations. Chickering and Gamson also stress the importance of encouraging contacts between students and faculty members, which might lead to a better integration of students and positively affect student involvement: Student–faculty contact is positively related to a range of educational outcomes (Tinto, 2012); and frequent contact between teachers and students is central to student engagement in academic activities (Xerri et al., 2018). Regarding the support variable, initiatives aimed at providing support to students have also demonstrated their ability to contribute to the improvement of students’ academic performance and retention. This is the case, for example, with supplemental instruction (see, e.g., Dawson et al., 2014; Malm et al., 2018; Paabo et al., 2021). Previous studies suggest that actions such as peer mentoring (Lane, 2020) or academic advising (Tippetts et al., 2022) might also be beneficial for student persistence, even though findings from previous research are not always consistent and even if more research is needed to establish the long-term effects of some of these initiatives (Lane, 2020). Regarding the institutional commitment variable, academic leadership is a necessary condition for creating quality learning environments (Bolander Laksov et al., 2014). There seems to be a relationship between leadership and student outcomes (Robinson et al., 2008). ⁸

Implications for Practice

Rather than testing the validity of Tinto and Pusser's (2006) model of institutional action for student success, the study proposed and tested a method to establish the profile of a particular department considering implementing actions to foster student success. This profiling was intended to develop a better understanding of the situation and the conditions for student success on which the department should focus its educational development efforts. The question is how, based on the findings of this study, the department in question could proceed to try to increase the performance indicator of the preparatory courses that are the subject of the study. Since each of the conditions for student success in this case study seemed to be related to course completion, an action plan should be implemented in this particular university department on a broad front and aim to enhance all the conditions, although it would probably be desirable to focus first on the factors with the highest effect sizes.

Another question is how to translate the results of this study into a set of concrete actions to promote student success. The findings of the study are, of course, determined by the way the scales of the questionnaire were constructed. For example, the questionnaire focused on two characteristics of feedback: its usefulness and its promptness. However, feedback is a complex construct and is offered in many different forms, which can have a wide range of effects on student learning and student success (Hattie & Clarke, 2018; Wisniewski et al., 2020). Similarly, student engagement (called involvement in Tinto and Pusser's model) is both an important predictor of student success and a complex, multidimensional process (Kahu, 2013) mobilizing emotions (e.g., anxiety, joy, pride, shame) as well as cognitive, metacognitive, and behavioral strategies (Dupont et al., 2015). Because of the complexity of these phenomena, the present study was supplemented with discussions with the teachers of the department. The discussions with the teachers were organized within pedagogical seminars on the basis of the results of the present study and Engel (2021). Participation was voluntary. Approximately one-third of the teachers in the department took part in the seminars. Here are some examples of suggestions made by teachers who participated in the seminars: increase the number of teaching hours as evidence of the department's commitment to student success; increase the number of “teacher hours” to allow for more frequent feedback; create study groups to provide academic support to students; and emphasize the importance of developing effective study techniques. The possibility of implementing supplemental instruction in the department was also discussed. In a next step, it would be useful to interview students who attended the preparatory courses of the department. In particular, it would be interesting to get the views of the students who did not pass the courses, as the perspectives of the students who dropped out are often missing in dropout studies (Agnidakis & Sundberg, 2018).

Another way to extend this study could be to seek faculty and student input based on different suggestions for actions that could be put in place to improve the different conditions for student success. As we saw in Figure 1, Tinto and Pusser (2006) provide examples of actions that shape the conditions for student success. For instance, assessment methods such as student portfolios (Parker White, 2004) are a way to enrich the feedback to students. Supplemental instruction is a type of peer support program with a potentially significant impact on student performance and student success (Dawson et al., 2014; Malm et al., 2012). ⁹ The teaching and administrative staff and the students could discuss the opportunity to set up such actions as well as the possible obstacles to their implementation.

Limitations

A first limitation of the study is that the response rate for the questionnaire was relatively low. A second limitation is that students who completed the course in which they were enrolled were over-represented in the study corpus, which may have biased the results of the study. This outcome could, however, be expected: The fact of participating in a survey, because of the effort that it requires, can be regarded as a proxy for student commitment (Stephenson et al., 2020), which is associated with academic performance.

Almost all the courses included in this study were conducted on campus prior to the Covid-19 pandemic. However, in mid-March 2020, Swedish universities decided to switch to distance teaching for health reasons. These circumstances are likely to have influenced the variables examined in this study. The lack of social contact between students may have led to a lower sense of community (Bolander Laksov et al., 2021), which, in turn, may have negatively affected student engagement. Distance learning also had a negative effect on the study's response variable: Mehic and Olofsson (2021) showed that online learning during the Covid-19 pandemic had a negative impact on course grades. This means that some students who did not complete their course during the pandemic might have completed it under normal circumstances.

Furthermore, student success in this study was equated with course completion, while the conditions for student success were measured based on student perception. Student perception is one way in which to approach, and attempt to identify, the conditions for student success in a department. However, students can perceive and be aware of only part of the practices of a department. It is also known that students do not always share the perception of teachers. To take one example, student and teacher perceptions of the role and efficacy of teacher feedback may differ (see, e.g., Schulz, 2001; Zhan, 2016).

The results of this study have shown a pattern: Students who complete their preparatory course estimate the conditions for success as being higher than students who do not complete the course. However, the study has only shown that there is a covariation; it has not demonstrated the existence of a causal relationship between the conditions for student success and course completion. There may be underlying variables or intermediate variables that explain student course completion. In order to try to show the existence of a causal relationship, it would be desirable to complete this study using other methods of investigation, such as interviews with students, as suggested above.

Recommendations If the “Profiling Method” Were to Be Used in Another Department

If this study were to be repeated, or if the profiling method were to be used in another department, it would be advisable to find ways to ensure a higher response rate. One way to operate would be to distribute the questionnaire in paper form during the semester at the end of a class.

Because of the complexity of the phenomena involved in the conditions for student success, it is important to make clear, during the discussions with students and teachers, how the study's latent variables were elaborated.

Another recommendation would be, if possible, to control for factors such as gender, results of previous studies, or socio-economic background.

Conclusion

Tinto and Pusser's (2006) conditions for student success made it possible to draw a dividing line between the language students who completed their courses and those who did not. The strength of the relationship between course completion and the different conditions for student success at the department where the study was conducted was measured with effect sizes. These effect sizes provided a “profile” of the department in question, allowing for the identification of areas on which interventions to promote student success should focus. Implications of these findings for educational development at the department were discussed. An interesting extension of this study would be to investigate other student populations in order to make comparisons with the findings of the present study and to measure the extent to which the conditions for student success vary across courses and departments.