Abstract
Few studies assessing stress symptoms are specific to medical education, and even fewer are designed to determine which stressors affect medical students. It was aimed to adapt the Medical Student Stress Factor Scale (MSSF) into Turkish and to provide a valid and reliable tool to determine stress caused by medical education in medical students. Between January and May 2024, 632 medical students participated. The Depression Anxiety Stress Scale-21 (DASS-21), the MSSF, and a sociodemographic data form were used. The MSSF is a 7-point Likert-type scale with 28 items. Although individual item scores range from 1 to 7, the total and subscale scores are calculated by averaging item scores, resulting in an overall score range from 1 to 7. Test-retest, Cronbach α, Davis technique, exploratory and confirmatory factor analyses (EFA and CFA), Pearson correlation, and descriptive analyses were used. The scale’s Cronbach’s alpha was .937, and the Kaiser-Meyer-Olkin (KMO) coefficient was .935. The sphericity results of the Bartlett test were x² = 9302.227, P < .001. In the CFA analysis, GFI = 0.814, CFI = 0.840, x²/df = 3.108, and RMSEA = 0.081 were determined. The MSSF total and subscale scores were positively correlated with all DASS-21 subscales (P < .001), supporting the construct validity of the Turkish version of the scale. It was found that the validity and reliability analyses of the Turkish version of the MSSF were satisfactorily met.
Health professions students exhibit significantly higher rates of mental illness compared to the general population. In Türkiye, awareness of stress in medical education is increasing, and studies have emphasized the need to address the causes of this stress.
The Turkish Medical Student Stress Factor Scale (MSSF) instrument showed strong psychometric qualities and is a valid and reliable tool for Turkish educators looking into and addressing stressors among medical students.
Since every nation and institution has unique cultural, social, and educational environment determinants, more validation of the MSSF instrument across a broad range of institutions is advised.
Introduction
Medical education is a challenging process that begins with the entrance to medical school, and the stress it creates on students is a subject being researched in many countries around the world and in Türkiye.1 -3 Several recent studies have reported that health professions students exhibit significantly higher rates of mental illness compared to the general population.2,4 In Türkiye, awareness of stress in medical education is increasing, and studies have emphasized the need to address the causes of this stress.5,6
Unfortunately, there is a 5.7% attrition rate in medicine and mental stressors are a major factor in this. 7 The most common stressors encountered by medical students can be listed as follows; inappropriate learning environment, 8 heavy workload,9 -11 lack of time, 10 competition among colleagues,6,7 financial concerns,9 -11 health problems, 9 and peer/faculty member relationships. 11
Stress has already been shown to impact students’ well-being and mental health, leading to high rates of depression, burnout, suicidal ideation, and medical school dropout.12 -14 Stress associated with medical education can have negative effects on patient care and lead to physician burnout. 15 Therefore, reliable assessment of medical student stress may reduce this negative impact and improve subsequent clinical experiences. 16 Considering that approximately 82% of medical students experience some degree of distress, vigilant assessment and monitoring of stress is necessary to avoid the challenges of the ubiquitous hierarchical medical education system. 17
In general, many studies conducted on university students and how the possible difficulties they encounter during their university education affect their mental health have reported results using longitudinal data.18,19 Indeed, these data show that issues such as mental health are dynamic processes. Therefore, although data are collected from a certain time in studies, it would be more valuable to present the results of the study by collecting data at certain time periods, considering that the mental state of students is a dynamic process.
Although many studies and tools worldwide have assessed stress symptoms in clinical, academic, and non-clinical populations, few are specific to medical education,20,21 and even fewer are designed to understand and weigh which stressors have the greatest impact on medical students’ mental health and quality of life. 22 In Türkiye, the Turkish validity of the Perceived Medical School Stress (PMSS) Scale, which was developed in 1984 to measure the medical school-specific stress perceived by medical students, was recently conducted. 3 This scale consists of 2 dimensions (Psychological Stress and Environment, Resilience and Expectations) and 13 items and only reveals the stress level of medical students. It does not evaluate the factors that cause this stress. The Medical Student Stress Factor (MSSF) Scale is a more updated scale developed in 2021 to investigate the most common stress sources faced by medical students and to what extent different stress factors affect their lives. 23 It also evaluates stress factors in 5 sub-dimensions in more detail than PMSS. MSSF has been used before 24 and has not yet been adapted to Turkish. We think that the scale will be the first tool that can be used to evaluate medical student stress sources in Türkiye. Therefore, our aim in this study was to adapt the Medical Student Stress Factor Scale (MSSF), originally developed by Damiano et al, into Turkish and to provide a valid and reliable tool to determine stress caused by medical education in medical students.
Methods
Participants
For this methodological 2-stage observational validity study, the sample group consisted of students studying at Sivas Cumhuriyet University Faculty of Medicine in the 2023 to 2024 academic year. It is advised to use Confirmatory Factor Analysis (CFA) in a separate dataset to confirm the validity of the structure discovered in the Exploratory Factor Analysis (EFA) of 1 dataset. 25 It is also known that the sample size should be 5 to 10 times the number of each item. 26 Since the total number of items in the scale was 28, it was aimed to reach at least 560 students, 280 students for EFA, and 280 students for CFA. No control group was included in the study. In the original study in which the scale was developed, no inclusion or exclusion criteria were reported for the scale other than being a medical student, volunteering to participate in the study, and signing the consent form. 23 Therefore, no other inclusion or exclusion criteria were applied in our study other than those listed. Studying at the faculty, 632 of 1300 students (participation rate 48.6%) volunteered to participate in the study. Data were collected face-to-face under the observation of at least 1 of the researchers between January and May 2024. The literature suggests that EFA and CFA analyses should be studied on different data sets. 25 We randomly divided our data set into 2 (316 students for EFA and 316 students for CFA) in the SPSS program. We calculated EFA and CFA on 2 different datasets.
Of the participants 370 were female students (58.5%), the mean age was 21.5 ± 2.4 (mean ± SD; min = 17, max = 35). The highest participation was from year 1 students (22.9%, n = 145). Students most frequently came from the Central Anatolia (45.6%, n = 288) and the Mediterranean (19.8%, n = 125) regions. The income of most students was equal to their expenses (55.7%, n = 352; Supplemental Table S1, available as online-only Supplemental Material).
Measures
To collect the study data, a questionnaire developed by the researchers to question the sociodemographic characteristics of the students, the Depression Anxiety Stress Scale-21 (DASS-21), and the MSSF (the Turkish version of the MSSF) were applied. The students’ age, gender, geographical region they came from, grade they were in, and their families’ average monthly income level (5 questions in total) were questioned as sociodemographic characteristics.
DASS-21 is a four-point Likert-type scale that contains 7 questions for each of the dimensions of depression, stress, and anxiety (21 questions in total). It is coded as 0 “does not apply to me,” 1 “somewhat applies to me,” 2 “generally applies to me,” and 3 “completely applies to me.” 27 Normal scores are less than 9 for depression, less than 7 for anxiety, and less than 14 for stress. 28 According to the Turkish validity and reliability study, the Cronbach alpha values of the DASS-21 subscales were found to be .87 for depression, .85 for anxiety, and .81 for stress. 29 Permission was obtained from the corresponding author via e-mail for the use of the Turkish scale.
The MSSF was developed by Damiano et al to investigate the most common stressors encountered by medical students and the extent to which different stressors affect their lives. The scale consists of 28 questions. 23 The scale rates the most common stressors experienced by medical students on a scale from 1 (not at all) to 7 (extremely distressing) for 28 situations. It assesses 5 different domains 23 : “Learning environment and academic performance” (questions 2, 9, 10, 13, 14, 15, 23, 24, 25, 27, 26, and 28); “Relationship” (questions 12, 16, 17, 18, and 19); “Health” (questions 11, 20, 21, and 22); “Lack of time” (questions 4, 5, and 6), and “Learning issues” (questions 1, 3, 7, and 8). To evaluate the scale, the scores of these 5 different domains and the overall scale score are calculated by averaging the scale items. No cut-off point was reported for the scale. The Cronbach’s alpha value for the entire scale was found to be .911. 23
Procedure
The cross-cultural adaptation of the scale was based on the recommendations of the World Health Organization and the literature review written on this subject.30,31 Figure 1 presents the intercultural adaptation stages applied in the research. In addition, the STROBE reporting instructions of the EQUATOR guide were followed in the study. 32 Permission was obtained via e-mail from the scale’s corresponding author, Dr. Giancarlo Lucchetti, to conduct the Turkish validity, and reliability study of the scale. Four independent native Turkish translators, 1 of whom was a professional translator and 3 of whom were medical academicians, translated the scale from English to Turkish. After the translation, the original text, translation, adaptation studies to other languages, and some studies where the scale was used were sent to 10 experts in the field. The experts were asked to evaluate each item of the scale 1 by 1 as (A) “Appropriate,” (B) “Needs minor revision,” (C) “Needs major revision,” and (D) “Not appropriate.” The Content Validation Index (CVI) was calculated using the Davis technique 33 in line with the responses from the experts. No item was removed from the scale based on the expert evaluations. The Turkish text was translated into English by an independent translator who had no knowledge of the scale, and this form was sent to the scale owner (Dr. Giancarlo Lucchetti) to obtain approval for any shift in meaning. A pilot application was conducted on 10 students to ensure that the scale was ready for data collection. The 46 students who agreed to participate in the test-retest phase of the study were retested 3 weeks later. Test-retest reliability analysis was performed. Situations such as participants not understanding enough, finding disturbing expressions, or requests for alternative expressions were reviewed and the final version of the scale was decided.

Intercultural adaptation stages applied in the research.
Ethical Aspects of the Study
Before the study, the participants were informed about the study, and their informed consent was obtained. Ethical approval (Decision no: 2023-11/31, Date: 16.11.2023) from the Sivas Cumhuriyet University Non-Interventional Clinical Research Ethics Committee and institutional permission (Date and Number: 14.12.2023-371033) were received for the study.
Statistical Analysis
For statistical analysis of the data, SPSS Version 23.0 (IBM Corp., Armonk, NY, USA) and IBM SPSS Amos 20 (IBM Corp., Armonk, NY, USA) package programs were used. Within the scope of reliability analysis of the scale, stability was evaluated with test-retest, internal consistency was evaluated with Cronbach’s alpha. Additionally, convergent validity, as part of construct validity, was assessed by examining correlations between the MSSF and the DASS-21 subscales. To test the validity of the scale, content validity was evaluated with the Davis technique. 33 In this technique, the number of experts who chose options (A) and (B) is divided by the total number of experts to obtain the CVI for the item, and >.80 is considered the suitability criterion for the item. We examined the construct validity of the scale with EFA. Using the direct oblimin, principal components, and EFA with oblique rotation approaches, the factor structure of 28 items was investigated. The suitability of the scale for factor analysis was evaluated with the Kaiser-Meyer-Olkin (KMO) sphericity method and the Bartlett test. The fit of the model was tested with CFA. The total variance values and factor loads, x2/df, comparative fit test (CFI), goodness fit test (GFI), non-normed fit index-Tucker Lewis index (TLI), and root mean square error of approximation (RMSEA) values were among the fit indices that were evaluated to assess the model fit of the CFA analysis.34,35 The relationship between the 2 scales was demonstrated using Pearson’s r correlation. Correlation analysis results were considered as a correlation coefficient (r) ≤.5 weak and .5 < r < 1 high correlation. 36 Descriptive statistical analyses were calculated for sociodemographic data and scale items. A P value of <.05 with a 95% confidence interval (CI) was considered statistical significance.
Results
Validation and Consistency Analyses of the MSSF Scale
The CVI for all items of the scale was above 0.80 (Supplemental Table S2, available as online-only supplemental Material).
The Cronbach α value calculated within the scope of the internal consistency analysis of the scale was found to be 0.937. The item-total correlation coefficients were between .315 and .713. Since there was no value below .30, it was decided that all items in the test exemplified similar characteristics and were appropriate to remain in the test. The reliability of the scale was also tested by the test-retest method. The test-retest correlation coefficients of the items of the scale ranged between .239 and .856 (Table 1).
Test-Retest and Cronbach’s Alpha Analyses of the Medical Student Stress Factor Scale.
Note. Cronbach’s alpha of total scale: .937.
>0.05 and **<.001.
The KMO coefficient calculated within the scope of the construct validity of the scale was determined as 0.935. Bartlett’s sphericity test results were χ2 = 9302.2, P < .001. Subscale Cronbach’s alpha values ranged between .721 and .920. Together, the 5 domains explained 60.3% of the variance (Table 2). The factor loads in Table 3 were expressed in accordance with the 5-factor model pattern matrix outcome in Figure 2. It was calculated as TLI = 0.822, GFI = 0.814, CFI = 0.840, x2/df = 3.108, RMSEA = 0.081.
Factor Loads Based on the Medical Student Stress Factor Scale Items’ Pattern Matrix From the Exploratory Factor Analysis.
Note. Principal components and direct oblimin methods were used.
KMO = Kaiser-Meyer-Olkin.
Correlations Between Medical Student Stress Factor Scale Dimensions and Depression, Anxiety and Stress Scale-21 Subscales.

Confirmatory factor analysis model of the MSSF.
Relationship Between MSSF Dimensions/Items and DASS-21 Subscales
Correlations between MSSF dimensions and DASS-21 subscales are given in Table 3. All dimensions of MSSF, for all subscales of DASS-21 were found to be positively correlated (P < .001 for each). Additionally, correlations between individual items of the MSSF scale and the DASS-21 subscales were analyzed. All items of MSSF, for all subscales of DASS-21 were found to be positively correlated (P < .001 for each).
MSSF Score Means and Main Stressors of Medical Students
The students’ MSSF general score mean was 3.7 ± 1.2. While the MSSF does not include predefined cut-off values, mean scores were used to descriptively indicate which factors were perceived as more or less stressful by the participants. Among the 5 dimensions, the highest mean score was observed in “Lack of time” (4.0 ± 1.7), followed by “Learning Environment/ Academic Performance” and “Health” (3.9 ± 1.5), while the lowest was in “Relationships” (2.9 ± 1.3). Regarding individual items, the most distressing stressors were extensive homework (4.6 ± 1.9) and exams/evaluations (4.5 ± 1.9), whereas issues such as sexual problems (2.4 ± 1.7) and family pressure (2.4 ± 1.8) were reported as less distressing.
Discussion
Using a tool like the MSSF to identify stress factors in various student groups, address student pressures, and stop further detrimental effects on mental health is crucial for medical schools. 37 Observing that stress factors are likely to reflect the particular cultural, social, and educational environment in the institution where the scale was developed, Roe et al—who were also medical students—recommended additional validation of the MSSF instrument in a variety of institutions. 38 In accordance with this information and suggestions, we translated the MSSF, which presents stress factors specific to medical students, into Turkish, and evaluated its validity and reliability.
The Turkish MSSF scale items’ CVI ranged from 0.90 to 1.00, as measured by the Karagöz and Kösterelioğlu Technique, indicating that the scale has adequate content validity. 39 Test-retest correlation, item-total correlation, and Cronbach’s alpha analyses were used to evaluate the reliability of the Turkish MSSF. Furthermore, the observed positive correlations between the MSSF and DASS-21 subscales interpreted as indicative of convergent validity and may support the construct validity of the Turkish version of the scale. Based on test-retest results, the scale items’ correlation coefficients ranged from .239 to .856. The overall scale’s test-retest reliability was determined to be .670. We found that the item-total correlation coefficients ranged from .315 to .713. Our results confirm that a value of .30 and above in the item-total correlation analysis indicates high item discrimination rates. It was discovered that only item 6′s test-retest Pearson’s r value was less than .30. 39 Nevertheless, there was no change seen when item 6 was eliminated from the scale; instead, the Cronbach’s alpha value decreased to .935. It was agreed to stay on the scale as a result.
The Turkish MSSF’s Cronbach’s alpha value was determined to be .937; in Damiano et al’ analysis, it was 0.911. 23 As a result, the Cronbach’s alpha value for the Turkish MSSF was over .80, indicating strong dependability. 40 Furthermore, we found that the subscale Cronbach alpha values in our analysis were higher than those in the original scale (which ranged from .650 to .907). 23
“Learning environment and academic performance,” “Relationship,” “Health,” “Lack of time,”, and “Learning issues” were the 5 factors that made up the EFA structure found in Damiano et al’ study and a total of 46.9% of the variance was explained. In our study, our KMO and Bartlett sphericity test results were 0.935 and 9302.2 (P < .001), respectively. The data was appropriate for factor analysis because the KMO was higher than 0.60 and the Bartlett test was significant.39,41 The EFA findings showed that the 5-factor model explained 60.3% of the overall variation. Consistent with the research of Damiano and colleagues, we called the 5-factor structure “Learning environment and academic performance,” “Relationship,” “Health,” “Lack of time,”, and “Learning issues.”
We applied CFA to test the model fit of this 5-factor structure. The fit index in the confirmatory factor analysis conducted on the sub-dimensions specified in the scale was assessed using the following values: x2/df, TLI, GFI, CFI, and RMSEA. A x2/df value of 3 and below is a very good indicator of model fit. Our result was 3.108. This result was evaluated as sufficient compliance. 42 The RMSEA measures the population’s approximate fit, it is focused on approximation-related discrepancies. The interpretation of the RMSEA threshold values has changed throughout time; in the past, values below 0.08 were regarded as good and values between 0.08 and 0.10 as medium. However, the allowable limit for this amount has been lowered by more modern methods. Early in the new millenium, values below 0.06 were seen as desirable in several research, but 0.07 was the threshold number in others. 43 Consequently, it may be concluded that the model is good when the RMSEA value is close to or more than 0.1, and that it is bad when it approaches or equals zero. GFI values typically range between zero and one with larger values indicating a better fit. 0.90 is indicative of a good fit relative to the baseline model, while values greater than 0.85 may be considered as an acceptable fit. While the GFI (0.814) and CFI (0.840) values in our CFA did not reach the conventional cutoff of 0.90 for a “good fit,” they are above 0.80, which is commonly accepted as indicating an adequate, or acceptable fit, particularly in scales with multiple factors and large sample sizes.344 546 Additionally, our model met other important fit indices, including x²/df = 3.108 (acceptable if <5) and RMSEA = 0.081 (acceptable if <0.08-0.10), supporting the overall structural validity of the model.44,45 The introduction of TLI statistics or NNFI (non-normed fit index) allowed for the elimination of sample size. Regarding the TLI threshold value, the literature contains a wide range of viewpoints. Apart from comparable values, like TLI >0.80, there are also high threshold values, like TLI >0.95. 46 The Turkish MSSF was found to have an acceptable level of compliance with the CFA fit index results based on these findings.
Since the MSSF is a relatively recently developed scale, we have not yet found a study that has adapted it to other languages. In this respect, our study is the first. The only study that used the scale was also conducted in Brazil. 24 Similar to our study, this previous study found that the MSSF dimension in which medical students scored the highest was “Lack of time,” while in which they scored the lowest was “Relationship.” 24 Furthermore, other global instruments designed to identify stressors among medical students comprise MSSF dimensions like “Learning environment/academic performance,” “Relationships,” and “Health”; this indicates that certain stressors are significant globally.8,17,18
Upon examining the primary stressors experienced by medical students, we found that the first 5 stressors differed from the Brazilian population used to develop the scale. However, the stressors such as “Extensive homework,” “Exams/Evaluations,” “Fear of failing any course,” and “Grades,” which we identified as the first 5 in our study, were also at the top of the list in the study in question. 23 Previous studies47 -50 have also shown that exams are a significant cause of stress for medical students. Furthermore, in the Brazilian sample, 23 stressors like “Romantic relationships,” “Family relationships,” “Sexual issues,” and “Peer relationships” were also at the bottom of the list.
In our study, we performed corelation analysis between MSSF dimensions/items and DASS-21 subscales. According to Radcliffe and Lester, the primary causes of stress for undergraduate medical students are academic issues. 51 Furthermore, compared to the Brazilian sample, we discovered substantial correlations between many more MSSF items and DASS-21 subscales in our investigation. Compared to the Brazilian sample, we found a significant relationship in the items pertaining to students’ patient relationships. This finding could be explained by the fact that our study included a larger number of students enrolled in clinical internships. Indeed, Damiano et al noted that 1 potential limitation of their study could be the small number of students enrolled in clinical internships. 23 Medical students also reported that the fear of making mistakes or incorrect decisions during clinical rotations was a significant source of stress, as highlighted in the meta-analysis. 52
The following is a list of our study’s limitations: Given that it was conducted with participants from a single faculty, it might not accurately represent all medical students. There was no consideration given to the interval between the students’ exam periods and the completion of the survey. In actuality, stressors might differ between the non-exam and pre-exam periods. 53 It is recommended that future studies be implemented by considering the exam periods (separately in the non-exam period and pre-exam period).
Conclusion
Medical students’ most frequent stressors were related to their academic performance and learning environment. The Turkish MSSF instrument showed strong psychometric qualities and is a valid and reliable tool for Turkish educators looking into and addressing stressors among medical students. Since every nation and institution has unique cultural, social, and educational environment determinants, more validation of the MSSF instrument across a broad range of institutions is advised.
Supplemental Material
sj-docx-1-inq-10.1177_00469580251356131 – Supplemental material for Adaptation of the Medical Student Stress Factor Scale Into Turkish: Validity and Reliability Study
Supplemental material, sj-docx-1-inq-10.1177_00469580251356131 for Adaptation of the Medical Student Stress Factor Scale Into Turkish: Validity and Reliability Study by İrem Akova, Ezgi Ağadayı, Nagehan Ekici Koşaroğlu and Gamze Gündoğdu in INQUIRY: The Journal of Health Care Organization, Provision, and Financing
Footnotes
Acknowledgements
We thank the medical students who participated in the study.
Ethical Considerations
Ethical approval (Decision no: 2023-11/31, Date: 16.11.2023) from the Sivas Cumhuriyet University Non-Interventional Clinical Research Ethics Committee and institutional permission (Date and Number: 14.12.2023-371033) were received for the study.
Consent to Participate
The participants were informed about the study, and their informed consent was obtained.
Author Contributions
İA and EA were responsible for conceptualization, methodology, formal analysis, and writing—original draft and review and editing. NEK and GG were responsible for methodology, formal analysis, and writing—original draft and review and editing. All authors have read and approved the final version submitted and take public responsibility for all aspects of the work.
Tracked Changes or Comments Where the Poster’s Name Is Listed
This study was presented as an abstract at the 3rd Black Sea Family Medicine Congress (16-19 May 2024, Samsun, Türkiye) and received the first prize for oral presentation.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
