Development and Validation of the Pre-service Teacher Competency Scale in an Online Learning Environment Using the Scenario Method

Abstract

The rapid shift to online learning during the COVID-19 pandemic highlighted the need to equip future educators with essential skills for an online learning environment. This study aims to address this gap by developing and validating a scale to measure pre-service teacher competency in an online learning context using the scenario method. After a comprehensive literature review, seven competencies of pre-service teachers in an online learning environment were constructed: design/planning competency, social competency, instructive competency, technological competency, management competency, positive teacher attitude competency, and learning competency. The Pre-service Teacher Competency in Online Learning Environments Scale was developed with 11 scenarios and 34 items and refined based on feedback from focus group interviews with six pre-service teachers and five in-service teachers to ensure face and content validity. The scale was then validated through confirmatory analyses and reliability analyses with 579 pre-service teachers, demonstrating good construct validity and reliability. This study contributes to teacher education by providing a validated tool that can help evaluate and enhance pre-service teacher preparation programs for online and hybrid learning environments.

Keywords

pre-service teacher teacher competency teacher education Korean education scale development

Introduction

The COVID-19 pandemic triggered a rapid and global shift to online learning, transforming the educational landscape. While traditional in-person classes had previously dominated, the pandemic necessitated a pivot to virtual classrooms, and even post-pandemic, a trend toward blended learning—combining face-to-face and online formats—has emerged and is likely to continue (Muthuprasad et al., 2021). However, this shift was initially met with widespread confusion among both students and teachers due to rarity of, online education was uncommon before COVID-19 (Godber & Atkins, 2021). This change highlighted the need for teacher education institutes to prepare future educators for the realities of online and hybrid learning environments.

The unique position of pre-service teachers—university students who are building foundational teaching skills and preparing for their future roles—makes it essential that teacher education institutes assess and develop competencies specifically suited for these digital and hybrid contexts. Pre-service teachers are distinct from in-service teachers in that they must navigate dual responsibilities as learners and aspiring educators (Zacharis, 2020). Currently, teacher competency scales exist but tend to focus on the latter, without addressing the unique characteristics and developmental needs of former (Zhumash et al., 2021). As university students, pre-service teachers require competencies that support their learning within the academic context and their motivation to teach, distinct from the needs of in-service teachers who already have professional experience (Amrein-Beardsley, 2020).

Moreover, most existing competency scales do not incorporate the rapidly evolving demands of online education, limiting their applicability in a post-pandemic context where hybrid learning is becoming more common. Although scales to measure teachers’ digital competency have been developed, most were created before the COVID-19 era and may not fully address the nuances of non-face-to-face learning situations (Hung et al., 2010; Pillay et al., 2007; Uzunboylu & Ozdamli, 2011). Furthermore, to the authors’ knowledge, no scale has been developed using the scenario method to measure practical competency (Bhagat et al., 2016; Reyes-Fournier et al., 2020). In addition, most studies on online education have focused on in-service teachers or learners, rather than pre-service teachers (Hung et al., 2010; Koc & Barut, 2016; Lin et al., 2015). Therefore, developing a competency scale tailored to the needs of pre-service teachers in online teaching environments is necessary to address these gaps.

This study addresses these needs by developing and validating a scale that can assess pre-service teacher competencies specifically for online learning situations. The proposed scale uses a scenario-based approach that reflects the challenges of digital learning environments and is expected to support teacher education institutes in assessing and fostering pre-service teachers’ readiness for blended learning.

This paper is organized as follows. First, we review the literature on pre-service teacher competencies to establish a foundational definition. Next, we outline the methods used to develop our scale, including scenario-based techniques and confirmatory factor analysis. In the results section, we present the validity of our findings and the criteria for selecting the final factor model. Finally, we conclude with the implications for future research and discuss the limitations of this study.

Theoretical Framework

Competency refers to the internal characteristics of an individual that enable effective job performance (Spencer & Spencer, 1993). As interest in competency increases, the attention of many researchers toward teacher competency also increases, which pertains to a teacher’s ability to perform a given task successfully in a particular situation (Gläser–Zikuda & Fuß, 2008; Goodyear et al., 2001; Huda et al., 2017; Kartal & Başol, 2019; Seferoglu, 2005; Selvi, 2010; Shulman, 1986). Table 1 summarizes previous studies related to in-service and pre-service teacher competencies.

Table 1.

The Constructs of In-Service Teacher and Pre-service Teacher Competency in Previous Studies.

	Authors	Competency
In-service Teacher	Gläser–Zikuda and Fuß (2008)	Clarity of instruction, Motivational competence, Diagnostic competence of social relations, Diagnostic competence of performance, Individual reference norm orientation, Teachers’ care
	Goodyear et al. (2001)	Content facilitator, Technologist, Designer, Manager/administrator, Process facilitator, Adviser/counselor, Assessor, Researcher
	Huda et al. (2017)	Learning competency, Social competency, Educational competency, Technological competency
	Kartal and Başol (2019)	Professional knowledge area: Content knowledge, Pedagogical content knowledge, Legislation and regulation knowledge
		Professional skills area: Planning education and teaching, Designing learning environments, Managing the learning and teaching processes, Assessment and evaluation
		Attitudes and values area: National, moral and international values, Attitudes toward students, Communication and cooperation, Personal and professional development
	Seferoglu (2005)	Instructional competencies, General social-cultural knowledge and skills, Field-specific knowledge and skills
	Selvi (2010)	Field competencies, Research competencies, Curriculum competencies, Lifelong learning competencies, Social-cultural competencies, Emotional competencies, Communication competencies
	Shulman (1986)	Pedagogical knowledge, Content Knowledge
Pre-service teacher	Choi et al. (2009)	Instruction, Assessment, Guidance, Management, Professional Development
	Kim et al. (2004)	Learner, Learning, Content, Curriculum, Instruction, Assessment, Management, Diversity, Professionalism, Reflection, Inquiry, Communication, Technology
	Meeus et al. (2009)	Teaching competencies: The teacher as supervisor of learning and development processes, Educator, Content expert
		Classroom manager, partnership competencies: The teacher as parents and caretakers, Colleagues and school management, Third parties
		Learning competencies: The teachers as lifelong learner, Innovator, Researcher

According to previous studies that defined teacher competency and explored teachers’ competency levels, Shulman (1986) mentioned pedagogical and content knowledge as teacher competencies. Selvi (2010) suggested seven sub-competencies to explain teacher competency: field competencies, research competencies, curriculum competencies, lifelong learning competencies, social-cultural competencies, emotional competencies, and communication competencies. Meanwhile, Gläser–Zikuda and Fuß (2008) explained teacher competency included clarity of instruction, motivational competence, diagnostic competence of social relations, diagnostic competence of performance, individual reference norm orientation, and teachers’ care. On the other hand, Seferoglu (2005) defined teacher competency with three sub-competencies: instructional competencies, general social-cultural knowledge and skills, and field-specific knowledge and skills. Additionally, Kartal and Başol (2019) distributed teacher competency into three areas and 11 sub-competencies, which are (1) professional area—content knowledge competency, pedagogical content knowledge competency, and legislation and regulation knowledge competency, (2) professional skills area—planning education and teaching competency, and designing learning environments competency, (3) attitudes and values area—national, moral and international values competency, attitudes toward students competency, communication and cooperation competency, and personal and professional development competency. Also, Goodyear et al. (2001) presented teachers’ roles in an online learning situation; a teacher should help students’ learning as a content facilitator, technologist, designer, manager, administrator, process facilitator, advisor, counselor, assessor, and researcher. Moreover, Huda et al. (2017) suggested learning competency, social competency, educational competency, and technological competency as required for teachers in the Big Data era. In sum, previous studies indicated that the competencies required of teachers include class management and building relationships with students.

As teacher competency is gaining interest, competencies for pre-service teachers attending teacher training institutions are also attracting much attention, and various studies have been conducted (Choi et al., 2009; Kim et al., 2004; Meeus et al., 2009). Choi et al. (2009) suggested instruction, assessment, guidance, management, and professional development as pre-service teachers’ competencies. Additionally, Meeus et al. (2009) explained that pre-service teachers should have teaching competencies as supervisors of the learning and development process, educators, content experts, and classroom managers; they should also have partnership competencies. Kim et al. (2004) suggested 13 dimensions as criteria of competent pre-service teachers: learner, learning, content, curriculum, instruction, assessment, management, diversity, professionalism, reflection, inquiry, communication, and technology. Based on these previous studies, pre-service teachers’ competencies include class conducting competency and managing student competency. Some literature (Kim et al., 2004) also suggested learning competency as a pre-service teacher competency.

As such, various studies have measured the competency levels of pre-service teachers, but most of those studies used teacher competencies, not pre-service teachers (Eren et al., 2015; Ersin et al., 2020; Tang et al., 2016; Yusuf et al., 2017). In addition, previous studies (Ersin et al., 2020; Kim et al., 2004; Yusuf et al., 2017) on pre-service teachers’ competencies did not consider online learning situations; thus, their limitation is not reflecting the rapidly changed educational environment in the post-COVID-19 era.

Pre-service teachers are students cultivating the necessary teacher competencies before becoming teachers; thus, they must have competencies as learners and teachers. They need teachers’ competencies to provide better education and learners’ competencies as students in a teacher training institution. However, most previous research did not reflect those pre-service teachers’ competencies required as a learner (Eren et al., 2015; Ersin et al., 2020; Tang et al., 2016; Yusuf et al., 2017). Therefore, this study defines pre-service teachers’ competencies to include teacher and learner competency, reflecting their unique characteristics.

This study focuses on the competencies of pre-service teachers especially required in an online learning environment. Teachers have experienced considerable difficulties from the sudden transition to online classes due to the COVID-19 pandemic. Therefore, we can assume that the competencies required in an online learning environment differ from those needed in previous face-to-face teaching situations (Alea et al., 2020; Fauzi & Khusuma, 2020; Pokhrel & Chhetri, 2021). It seems that online classes used in flipped learning before the COVID-19 pandemic, more commonly after the pandemic, with the development of new technologies such as Metaverse and Zoom (Abdous, 2011). Therefore, it is necessary to define the required competencies and develop a measurement scale that identifies their current level to improve pre-service teachers’ competencies in this online learning environment.

Definition of Pre-service Teacher Competency in Online Learning Environment

This study defines pre-service teacher competencies for online learning environments by drawing on findings from previous studies and focus group interviews with both teachers and teacher candidates. Highlighting the need to define these competencies specifically for pre-service teachers, this study addresses a gap in the research, as prior studies have largely focused on in-service teacher competencies and have not fully considered the unique demands of online education, particularly in the post-COVID-19 context. Since pre-service teachers must develop skills both as learners and as future educators, it is crucial to identify and assess these competencies to better equip them for the evolving educational landscape. Given the shift to online classes and the increasing integration of new technologies, establishing a measurement scale to evaluate these competencies has become essential. Accordingly, we selected competencies needed by in-service teachers from the literature and added two additional competencies centered on learning attitudes to emphasize the importance of a learner’s approach.

Design/Planning Competency

Most of the literature mentioned designing/planning for the class as one of the most important teacher competencies. For instance, Han and Nam (2020) stated that online class teachers should set a goal for their classes, construct the content to teach, and inform students about learning. Also, teachers should plan for proper evaluation methods before the course starts. In Jeong (2020)’s paper, he stated that teachers in the COVID-19 era need to plan for their online classes and operate it as planned. Welch (2014) tried to develop a scale for virtual teaching dispositions and indicated that teachers must improve their design and organizing skills by interacting with others. Those are things teachers must do before online classes. In addition to Han and Nam (2020), Jeong (2020), and Welch (2014), König et al. (2020) and Ally (2019) also thought of design and planning competencies as critical teacher competencies. Competencies to operate classes and protect digital information, plans for making media materials and producing materials for class, and facilitating a customized class for each student are examples of design/planning competency (Ally, 2019; König et al., 2020). Moreover, Baek and Kim (2020) developed a scale for pre-service teachers anticipating them to be good teachers in the future. They mentioned that teachers must have the competencies to plan a course creatively, develop and operate a curriculum, and produce ICT content before the course.

Thus, we defined design and planning competency as actions undertaken before the course to achieve the educational goal. There are two types of actions undertaken before class. The first is cooperation and consultation with stakeholders who can help teachers to achieve academic goals: parents, fellow teachers, school administrators, etc. The other is professional preparations for classes, including lecture planning, preparing class materials, checking students’ academic levels, figuring out students’ features, and confirming that students have all the equipment necessary for classes. By doing such actions, pre-service teachers can prepare for a well-structured class.

Social Competency

Many researchers emphasized the importance of social relationships between teachers and their students. For example, Yoon (2021) mentioned that digital interaction is one of the teachers’ competencies for digital literacy education. König et al. (2020) and Ally (2019) also stated that teachers need to communicate with students well in online classes and solve cyberbullying by interacting with students. Similarly, Bigatel et al. (2012) said that teachers could create a peaceful environment and prevent conflicts between pupils by modeling anticipated behaviors for students. Oh (2020) studied teachers’ competencies for distance learning and emphasized that teachers imperatively promote students’ interaction and give students immediate feedback. For instance, Kabilan (2004) anticipated teachers to interact with students, and Goodyear et al. (2001) even said that teachers must counsel students. Baek and Kim (2020) noted that pre-service teachers need social interaction and sympathy.

This study defines social competency as teachers’ actions and attitudes to build rapport with their students in a virtual educational environment. Also, pre-service teachers with high social competency tend to have skills in delivering their emotions and ideas properly by using non-verbal and verbal expressions. In addition, they can keenly observe interactions between students. Through this, they can intervene or mediate relationships between students and prevent problems that might occur, such as cyberbullying.

Instructive Competency

As a teacher, one should have strong content knowledge and the ability to teach well. Therefore, prior studies underlined the importance of teachers’ instructive competency. For example, Han and Nam (2020) and Welch (2014) stated that teachers must have knowledge about the content they will teach. In addition, Martin et al. (2019) emphasized teachers’ knowledge about content and learning to deliver content efficiently. Jeong (2020), who executed research about education in the COVID-19 era, highlighted teachers’ understanding of remote learning as a key knowledge area. Goodyear et al. (2001) even said that teachers must become researchers studying students and content. Moreover, Baek and Kim (2020) widened the range of knowledge teachers should possess. According to their research, the knowledge that teachers should know includes multicultural knowledge, global issues, environmental issues, educational policy, pedagogical knowledge, and content knowledge.

To summarize, instructive competency comprises pedagogical knowledge, content knowledge, and methodological knowledge. In our study, pedagogical knowledge refers to knowledge about the traits of students and the way to deliver a lecture efficiently. A pre-service teacher with a high level of pedagogical knowledge can arouse students’ interest and encourage students to participate in the class. Content knowledge is specific information about subjects teachers should cover in their classes. A pre-service teacher with content knowledge has professional knowledge related to the topic. Moreover, methodological knowledge is concrete knowledge needed in an online class, such as knowledge about software utilized in the virtual class. Finally, a pre-service teacher with methodological knowledge can construct and deliver virtual classes efficiently by applying various teaching methods, including flipped classes, blended classes, recorded classes, and real-time classes, etc.

Technological Competency

Teaching in an online learning environment requires teachers to have sufficient technical ability. Many previous studies mentioned teachers’ technological competency as an essential factor for effective teaching; for example, Bigatel et al. (2012) and Kabilan (2004) stated that technological competency is a critical aspect of online course instructor preparation. According to Goodyear et al. (2001), the technologist is one of the teacher’s roles involved in online teaching, which is concerned with making or helping make technological choices that improve the environment available to learners. Ally (2019) conducted a study about the competency profile of digital and online teachers in future education, emphasizing teachers’ ability to use digital technology and that the teacher should be digitally literate to use technology to deliver and support education. Han and Nam (2020) also found that utilizing technology is one of the core teaching competencies to improve the quality of teaching online classes.

Therefore, technological competency means the required skills to deliver a lecture efficiently or conduct teachers’ various tasks using devices and educational software. It contains the ability to adapt to newly released technologies in a short period. Also, a pre-service teacher with technological competency should be conscious of copyright ethics and use materials legally.

Management Competency

A teacher should have online classroom management strategies that directly affect the student’s ability and the teacher’s teaching ability. Han and Nam (2020) and Oh (2020) emphasized teachers’ abilities to manage an online classroom, such as giving students feedback, supervising, and adjusting the learning process. König et al. (2020) also mentioned the importance of operating and applying classroom structure plans predetermined before the actual course during class time. According to Goodyear et al. (2001), online teachers should be assessors concerned with providing feedback, grades, and validation of learners’ work, and manager-administrators concerned with the issues of learner registration, security, record keeping, and so on. In addition, Jeong (2020) stated that teachers should have the competency that enables effective online classroom management to enhance the quality of distance teaching.

In this study, the management competency includes teachers’ actions to carry out the plans established before the course during class time. This competency consists of actions necessary for the successful completion of the teaching and learning experience. Teachers with management skills can appropriately cope with and solve issues that occur during class. To handle the virtual classroom during the whole class, teachers should manage communication channels and spaces and supervise the ongoing and virtual process (Guasch et al., 2010). Also, teachers should have time management skills which refer to constructing and implementing timetables, distributing subjects and workloads, planning lessons, and allocating and using time effectively (Sahito & Väisänen, 2017). In addition, teachers with management competency can grasp the class progress by conducting formative and summative assessments and giving feedback to students.

Positive Teacher Attitude Competency

Positive teacher attitude means job-related attitudes, including teacher self-efficacy and job satisfaction (OECD, 2019). A positive teacher attitude is essential because it can prevent pre-service teachers from burnout and dropout (Hong, 2010; OECD, 2019). Researchers found a negative correlation between the degree of burnout and teachers’ attitudes toward change (Sokal et al., 2020). Also, the Teaching and Learning International Survey 2009 (OECD, 2009) found a close link between teacher attitudes and strategies for coping with daily professional challenges, including general well-being. Teachers shape students’ learning environments and influence student motivation, achievement (OECD, 2019), and emotion (Tang & Hu, 2022). Thus, a positive teacher attitude helps teachers work effortlessly and be satisfied with their work; therefore, teachers with positive attitudes tend to be successful (Begum & Khan, 2015).

Pre-service teachers with this attitude have a strong willingness to become a teacher and a high expectation to become one, so those with positive teacher attitudes strive to become a teacher even if there are expected difficulties. In addition, pre-service teachers with a positive attitude have distinct views of teaching. The view of teaching is a framework of perception that shows how teachers see and understand teaching. In other words, the view of teaching is about how society thinks about the epitome of teaching and is a concept related to how teachers grasp the teaching profession (Kim, 2009).

Learning Competency

Learning competency is essential for pre-service teachers because they must study and learn to fulfill all the teacher certification requirements in their countries. The learning competency includes how students use different cognitive and metacognitive strategies and manage various resources (Lee et al., 2011; Pintrich et al., 1991). Lee et al. (2011) distinguished the learning competency into two sections. The cognitive and metacognitive strategies section includes items regarding concentration strategies, note-taking, memory strategies, and regulating efforts. The student management of different resources section includes items concerning managing the learning environment, using resources and asking for help, and preparing for a career.

Learning competency includes the willingness to learn, getting motivated to learn, and having a passion for learning as a college student. This competency consists of two sub-domains: “in class” and “out of class.” The variables of an “in class” area are using note-taking and concentration strategies. In contrast, the “out of class” area includes regulating efforts, managing the learning environment, using resources and asking for help, and preparing for a career.

Method

We conducted this study in three phases. First, we developed a scale draft using the scenario method based on the definition of competencies of pre-service teachers in an online learning environment. The scale draft consisted of 13 scenarios and 69 items. In the second phase, we confirmed content and face validity through and surveys with both pre-service and in-service teachers, who participated in focus groups conducted to define pre-service teacher competencies. After completing these steps, we prepared the final scale. Lastly, we validated the scale with a survey of 579 pre-service teachers in Korea. Reviewers of Ewha IRB confirmed the acceptability of this study.

Phase 1: Survey Development

This study used the scenario method, a scale development method that presents various situations that may occur to subjects in the future. There are several advantages to using this method. First, it helps subjects to understand the question and immerse themselves in the test easily (Banerjee et al., 1993). Second, it takes a lot of cognitive effort to understand the question and, thus, prevent response biases (Baek & Kim, 2020; Deane et al., 2019). Also, subjects can predict possible problematic situations through the various conditions presented in the scenarios. Thus, responding to the scale can provide an opportunity to practice for problem situations (Examsoft, 2021: 3). Therefore, the researchers deemed the scenario method essential for measuring the competencies of pre-service teachers in online learning environments because pre-service teachers can anticipate and experience situations they may encounter as teachers through the scale, enabling them to learn and reflect on coping strategies. Additionally, by employing the scenario method, pre-service teachers are encouraged to imagine and immerse themselves in school teaching situations they have not personally experienced, thereby reducing biases resulting from a lack of understanding when responding to the items.

In this study, we presented the scenarios to evaluate the seven sub-components required of pre-service teachers based on previous research analysis. These scenarios depicted online learning situations that necessitate each competency. Specifically, we developed virtual problem situations within the context of online teaching, representing scenarios where in-service teachers could encounter situations related to online teaching competencies. The purpose was to assess whether pre-service teachers could effectively demonstrate their online teaching competencies in these scenarios.

Phase 2: Item Analysis

We verified the content and face validity of the items with in-service and pre-service teachers. The investigation of the content validity and face validity included determining (1) whether the situations presented in the scenarios were likely to occur in the educational field and (2) whether the content of the items presented as responses to the scenarios was reasonable. In cases of low validity of the presented situations, we asked participants to provide a reason. The final questionnaires comprehensively considered the focus groups results, content validity, and face validity results conducted for in-service and pre-service teachers.

At this time, we determined the content validity of the study based on the responses of in-service teachers and the face validity based on the responses of pre-service teachers. We used this approach because in-service teachers have experience both experience in online classes in the actual educational field and, at the same time, have educational expertise. We assessed the suitability and appropriateness of the content and words by asking pre-service teachers. Based on the focus groups, we modified our scale according to the experiences of the interviewees. The specific modifications are presented in the results section.

Participants for Focus Groups

Considering the characteristic of the scenario method that induces subjects to immerse themselves in the problematic situation, we conducted focus groups to determine whether the scenarios comprised situations likely to be experienced by pre-service teachers and whether they were suitable for measuring competency. In particular, we conducted focus groups with each pre-service and in-service teacher since the situations that could be experienced in the school field and college learning situations might differ. Previous studies conducting focus groups showed that two or three groups were sufficient (Guest et al., 2016). Krueger (1994) indicated that the research needed at least three groups, while Stewart and Stewart (1990) stated no general standard set for the number of groups.

Several studies have suggested criteria for the number of participants in focus groups. Previous research indicates that six to eight participants are sufficient for conducting focus groups effectively (Greenbaum, 1988; Rabiee, 2004). Additionally, evidence suggests that smaller groups tend to encourage more active interaction among participants (Carey, 1994; Morgan, 1996). Due to the nature of the focus group method, even a small number of participants—between four and ten—can yield meaningful research results when carefully selecting participants relevant to the research topic (Howard et al., 1989; Kitzinger, 1995). For this study, we selected five in-service teachers and six pre-service teachers.

The standard for selecting in-service teachers as research participants were (1) teachers who became public school teachers by passing the national teacher certification tests or private school teachers who passed private schools’ own tests at a similar level to the national one, (2) conducted online classes, and (3) had more than 1 year of teaching experience. We selected three novice teachers with less than 5 years of teaching experience (OECD, 2019) and two experienced teachers with more than 5 years of experience. The standards for selecting pre-service teachers as research participants were college students wanting to become teachers and experienced teaching practice or educational volunteering. Tables 2 and 3 summarize information about the in-service and pre-service teachers participating in the focus groups.

Table 2

In-service Teachers Who Participated in the Focus Groups.

Participant	Gender	School Level	School Type	Year of Employment	Teaching Subject	Education Level
A	Female	Elementary School	Public	2015	All Subjects	Master’s Degree
B	Female	Middle School	Public	2017	History, Moral Education	Master Course
C	Female	Elementary School	Public	2021	All Subjects	Bachelor’s Degree
D	Male	High School	Private	2013	Geography, Social Sciences	Bachelor’s Degree
E	Female	Middle School	Private	2020	History, Social Sciences	Bachelor’s Degree

Table 3.

Pre-service Teachers Who Participated in the Focus Groups.

Participant	Gender	Undergraduates from the College of Education	Major	Teaching Subject	Types of Teaching Experience
F	Female	No	Chinese	Chinese	Volunteer Teaching
G	Female	Yes	Biology	Biology, Physics, Mathematics, Geoscience	Volunteer Teaching
H	Female	Yes	Social Sciences	Moral Education, Korean, English	Volunteer Teaching
I	Female	Yes	Mathematics, Education	Mathematics	Teaching Practice
J	Female	Yes	Mathematics, Education	Mathematics	Teaching Practice
K	Female	Yes	Education	Moral Education	Teaching Practice

Phase 3: Survey Validation and Reliability

After confirming the items’ face validity and content validity, we validated the items through a survey targeting pre-service teachers; our goal was to examine the items’ reliability and construct validity. The descriptive statistics of the survey respondents are in Table 4. We used the R software lavaan package (Rosseel, 2012) to verify the construct validity and Cronbach’s $α$ package (Taber, 2018) to verify the reliability. Since the survey result satisfied the normality assumption, we conducted a CFA with maximum likelihood estimation on each sample using the Lavaan package (Rosseel, 2012) in R to test the competing models for the structure of the scale.

Table 4.

Survey Respondents Descriptive Statistics.

Variable	Classification	Frequency	Percentage (%)
Sex	Female	425	73.40
Sex	Male	154	26.60
Age	19–20	104	17.96
	21–22	200	34.54
	23–24	177	30.57
	Above 25	98	16.93
Grade	Freshman	80	13.82
	Sophomore	123	21.24
	Junior	176	30.40
	Above Senior	200	34.54
Location of a teacher training institution	Seoul (the biggest city in Korea)	315	54.40
	Metropolitan Cities	132	22.80
	Small Cities	126	21.76
	Rural Area	6	1.04
Type of teacher training courses	Pre-service teachers majoring in secondary school teacher education	314	54.23
	Pre-service teachers majoring in primary school teacher education	165	28.50
	Pre-service teachers not majoring in teacher education but enrolled in teacher education courses	100	17.27
Educational volunteer work hours	None	65	11.23
	More than 1 hr, less than 10 hr	110	19.00
	More than 10 hr, less than 30 hr	180	31.09
	More than 30 hr, less than 60 hr	109	18.83
	More than 60 hr	115	19.86
Type of teaching practice	None	222	38.34
	Face-to-face teaching practice	168	29.02
	Online teaching practice	50	8.64
	Blended teaching practice (face-to-face and online teaching practice)	139	24.01
Whether to become a teacher after graduation	Yes	521	89.98
Whether to become a teacher after graduation	No	58	10.02

Participants

We surveyed 600 pre-service teachers to verify the scale’s validity. In general, the sample size required to verify the scale’s validity is judged based on the number of items being measured or the absolute number of participants. According to Hair et al. (1995), in factor analysis, 20:1 is appropriate depending on n:p (the ratio of cases to the measurement variable). The absolute standard ranges are 50 participants (very poor), 100 (poor), 200 (fair), 300 (good), 500 (very good), and 1,000 or more (excellent) (Comrey & Lee, 1992). In this study, we individualized the scale using 34 items; thus, we finally surveyed 600 Korean pre-service teachers according to the criteria presented in previous studies. Among them, (1) health, nutrition, and counseling teachers (13 people) who do not conduct classes for students, (2) early childhood teachers (3 people) who should conduct face-to-face classes, (3) who did not respond to the teacher qualifications question (1 person), and (4) current teachers attending graduate schools (4 people) from the final analysis.

According to the Enforcement Rule of the Teacher Qualification Examination Ordinance by the Ministry of Education of the Republic of Korea, pre-service teachers must complete at least 60 hr of educational volunteer work to obtain teaching qualifications in kindergartens, elementary, secondary, and special schools. So, we asked respondents how much time they had spent on educational volunteer work.

The gender ratio of respondents was about 7:3. This might seem somewhat skewed. However, female teachers comprise 77.2% of elementary schools, 71.6% of middle schools, and 57.1% of high schools in Korea (Ministry of Education (South Korea) and Korean Educational Development Institute, 2022). Therefore, the gender ratio of respondents reflects that of current Korean teachers.

Results

The final scale items comprehensively considered the focus group results, content validity, and face validity results of information provided by in-service and pre-service teachers.

Content and Face Validity

Based on the criteria presented by Fehring (1987), it is desirable to exclude items with a content validity index (CVI) of .50 or less; the content validity of seven items and four scenarios in the CVI analysis was less than .50. Among them, we modified one item (1-2) to measure the competency of pre-service teachers accurately based on the responses of in-service teachers. We used the remaining six items (4-1, 6-4, 7-1, 7-2, 9-3, 11-1) and four scenarios (5, 6, 10, 11) in the final scale without modifications. The teachers responded that they gave low scores to those items and scenarios because they believed the behaviors required in the items were very challenging to implement or the situations presented in the scenarios occurred infrequently. However, despite their difficulty or infrequency, they are crucial for measuring high pre-service teacher competency, so we decided to preserve them.

Additionally, based on the responses of in-service teachers, we included three other items necessary for measuring the competency of pre-service teachers. As a result, we finalized 34 items and 11 scenarios (see Appendix A). The final scale’s descriptive statistics and correlations are in Appendix B. Table 5 describes the CVI values of scenarios and items.

Table 5.

CVI Values of Scenarios and Items.

Scenario	CVI		Item	CVI
Scenario	In-service	Pre-service	Item	In-service	Pre-service
1	0.80	1.00	1-1(tec)	1.00	0.67
			1-2(ins)	0.40	0.83
			1-3(ins)	0.80	0.67
			1-4(ins)	0.60	0.67
2	1.00	1.00	2-1(ins)	0.80	0.67
			2-2(des)	1.00	1.00
			2-3(des)	0.80	1.00
3	1.00	0.67	3-1(des)	0.80	0.83
			3-2(des)	1.00	1.00
			3-3(des)	0.60	1.00
4	1.00	0.83	4-1(man)	0.20	0.67
			4-2(man)	1.00	1.00
			4-3(tec)	0.60	0.83
5	0.40	1.00	5-1(tec)	1.00	1.00
5	0.40	1.00	5-2(tec)	0.60	1.00
6	0.40	1.00	6-1(lea)	0.80	0.67
			6-2(lea)	newly added	newly added
			6-3(pos)	newly added	newly added
			6-4(pos)	0.40	0.67
7	1.00	1.00	7-1(pos)	0.20	0.50
			7-2(lea)	0.40	1.00
			7-3(lea)	0.60	0.83
8	0.80	0.67	8-1(pos)	0.60	0.67
			8-2(lea)	0.60	0.83
			8-3(pos)	0.80	0.67
9	0.60	0.83	9-1(soc)	1.00	0.83
			9-2(soc)	0.60	0.83
			9-3(soc)	0.40	1.00
10	0.00	0.50	10-1(soc)	1.00	1.00
			10-2(soc)	1.00	1.00
			10-3(soc)	0.80	0.67
11	0.20	0.67	11-1(man)	0.20	1.00
			11-2(man)	0.60	0.83
			11-3(soc)	newly added	newly added

Note. Columns in boldface represent the scenarios and items modified or remaining despite low content validity. In this table, the first three letters of each competency are for abbreviation. For example, “tec” means the item belongs to the technological competency.

Model Selection

The next step involved confirmatory factor analysis (CFA) to test the model fit of a series of models. We evaluated the model fit using a multi-index strategy, as recommended by several methodologists (e.g., Byrne, 1989; Hu & Bentler, 1999; Jackson et al., 2009; Thompson & Daniel, 1996).

Further, we used the $x^{2}$ statistic as the absolute fit index with associated degrees of freedom and p-value. The $x^{2}$ /df value is appropriate between 1 and 2 or 1 and 3 (Byrne, 1989; Carmines & McIver, 1981). We also used the sample-size adjusted BIC (SABIC) as another absolute fit index; relatively lower SABIC values indicate better model fit. The comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the Tucker–Lewis index (TLI) were incremental fit indices used in our study. CFI and TLI values greater than 0.8 and RMSEA values lower than 0.5 indicate a reasonable fit (Bagozzi & Yi, 1988). Table 6 presents a summary of the fit indices of the competing models. Figure 1 visually represents hypothesized and competing models of our study.

Table 6.

Goodness-of-fit Indices of the Models.

Indices	Models
Indices	A	A’	B	C
$x^{2}$	1,474.634	3,964.890	1,505.213	1,312.371
$x^{2}$ /df	2.914	6.766	2.895	2.662
CFI	0.850	0.503	0.847	0.873
TLI	0.834	0.435	0.835	0.856
ABIC	42,628.743	48,992.995	42,614.708	42,507.907
RMSEA	0.057	0.100	0.057	0.054

Note. Model A’ is a variation of Model A allowing the correlation between the errors of the indicators within the same scenario.

Figure 1.

Three competing CFA models.

Model A is this study’s base model hypothesized that each indicator is directly specified to load in seven factors. Since the factors commonly constitute a concept of pre-service teacher competencies, the correlation between factors is allowed.

We tested Model B, a higher-order CFA model because the correlation between factors is high. Also, the items can be affected by not only pre-service teacher competencies but also a situation interpretation competency because the participants may have interpreted the scenario-based survey differently (Rocha et al., 2014). Therefore, we also tested Model C, a bifactor model. The results showed acceptable goodness-of-fit indices for all three models (see Table 6). However, there was very little difference among Model A, Model B, and Model C. Therefore, we selected the more parsimonious Model A and rejected Model B and Model C.

Model A’ is a variation of Model A with 46 additional parameters. Model A’ allows the correlation between the errors of the indicators within the same scenario, so we can check whether there is a scenario effect. We found no dramatic change; therefore, the scenario effect was not strong, and we rejected Model A’.

Consequently, we chose Model A as this study’s final model. Table 7 summarizes the factor correlations.

Table 7.

Inter-Correlation Matrix between Factors and Total Score.

	1	2	3	4	5	6	7
Design/planningCompetency
Socialcompetency	0.672*
Instructivecompetency	0.604*	0.559*
Technologicalcompetency	0.663*	0.650*	0.575*
Managementcompetency	0.554*	0.539*	0.557*	0.606*
Positive teacherattitude competency	0.635*	0.660*	0.583*	0.652*	0.569*
Learningcompetency	0.652*	0.646*	0.570*	0.612*	0.552*	0.716*
Total score	0.846*	0.848*	0.765*	0.834*	0.744*	0.842*	0.827*

Note.*p < .001 The correlations between each factor and the total score of pre-service teacher competency ranged from .744 to .848, and the inter-correlations between the seven factors ranged from .539 to .716 (see Table 8). These correlations indicate that the seven factors had something in common but varied in sufficient ways. Also, the relatively lower correlation coefficient between factors was high.

Table 8 summarizes the factor loadings and Cronbach’s $α$ coefficients of our final model. The factor loadings are the regression coefficients (i.e., slopes) for predicting the indicators from the latent factor. Generally, the higher the factor loading, the better; typically, loadings below 0.30 are not interpreted. When the items have different frequency distributions, Tabachnick and Fidell (2007) follow Comrey and Lee (1992) in suggesting using more stringent cut-offs going from 0.32 (poor), 0.45 (fair), 0.55 (good), 0.63 (very good) or 0.71 (excellent). In this study, all item loadings were greater than .42, which indicated that the items were related to each other as separate factors in the model (see Table 8).

Table 8.

Factor Loadings for Final Model (Model A) and Cronbach’s $α$ Coefficients.

Factors	Items	Unstandardized Factor Loadings	Standardized Factor Loadings	$R^{2}$	Cronbach’s $α$ if Item Dropped	Cronbach’s $α$ by Competency	Cronbach’s $α$ of Total
Design/ planning competency	2-2	1.000*	0.540	0.292	0.77	0.76	0.93
	2-3	1.210*	0.636	0.405	0.72
	3-1	1.392*	0.705	0.497	0.71
	3-2	1.335*	0.744	0.553	0.67
	3-3	0.969*	0.549	0.301	0.74
Social competency	9-1	1.000*	0.552	0.304	0.72	0.75
	9-2	0.862*	0.460	0.211	0.73
	9-3	0.913*	0.529	0.280	0.72
	10-1	1.359*	0.722	0.521	0.69
	10-2	1.124*	0.665	0.443	0.70
	10-3	0.768*	0.424	0.180	0.74
	11-3	0.954*	0.505	0.255	0.72
Instructive competency	1-2	0.924*	0.521	0.272	0.48	0.60
	1-3	1.000*	0.533	0.284	0.55
	1-4	0.748*	0.420	0.176	0.55
	2-1	1.137*	0.593	0.352	0.53
Technological competency	1-1	1.000*	0.598	0.357	0.60	0.65
	4-3	0.887*	0.500	0.250	0.64
	5-1	1.186*	0.610	0.372	0.52
	5-2	0.940*	0.571	0.326	0.57
Management competency	4-1	1.000*	0.428	0.183	0.49	0.55
	4-2	1.293*	0.611	0.373	0.45
	11-2	1.003*	0.449	0.202	0.51
	11-1	1.137*	0.456	0.208	0.48
Positive teacher attitude competency	6-3	1.000*	0.610	0.373	0.67	0.73
	6-4	0.787*	0.437	0.191	0.71
	7-1	1.130*	0.610	0.372	0.68
	8-1	1.454*	0.757	0.573	0.64
	8-3	0.924*	0.507	0.257	0.70
Learning competency	6-1	1.000*	0.706	0.499	0.61	0.68
	6-2	0.731*	0.503	0.253	0.63
	7-2	0.726*	0.488	0.238	0.63
	7-3	0.676*	0.523	0.273	0.62
	8-2	0.727*	0.485	0.236	0.66

p < .001.

A Cronbach’s $α$ coefficient of .70 and above is good, indicating that the items had sufficiently robust reliabilities for meaningful use in further analysis. However, Cronbach’s $α$ has some limitations. One of them is that scores with fewer items associated with them tend to have lower reliability. In this study, the Cronbach’s $α$ coefficients of each factor ranged from .55 to .76, with those factors below .70 had only four or five items each. The Cronbach’s $α$ coefficients are relatively low, not because items and factors are unreliable but because the number of items is small.

Table 9 shows the scale’s cut scores. We determined the score’s norm using the responses of 579 survey respondents, calculated with a z-score. If the z-score of each competency and the total score was 1 point or above (top 15.87%), the subject had an excellent level of competency. Students with a z-score of −1 or higher than −1 and lower than 1 (68.26%) are subjects with an average level of competency, and students with a z-score lower than −1 (bottom 15.87%) have a poor level of competency. These cut scores can be used for educational purposes to improve and identify the competencies of pre-service teachers.

Table 9.

Cut Scores for the Pre-service Teacher Competency Scale in an Online Learning Environment.

Competencies	Poor	Average	Excellent	Minimum Score	Maximum Score
Design/planning competency	5–17	18–23	24–25	5	25
Social competency	7–24	25–32	33–35	7	35
Instructive competency	4–14	15–18	19–20	4	20
Technological competency	4–13	14–18	19–20	4	20
Management competency	4–13	14–17	18–20	4	20
Positive teacher attitude competency	5–17	18–22	23–25	5	25
Learning competency	5–17	18–22	23–25	5	25
Total	34–125	126–155	156–170	34	170

The goal of this scale is to evaluate and improve the competency of pre-service teachers. Achieving this goal requires evaluating each competency. This is because it is not desirable to judge the level of competency only by the total score of the scale, given that the competency of pre-service teachers consists of seven factors. Therefore, teacher training institutions can use the scores to diagnose each competency of pre-service teachers, modify and develop curriculum and other activities for individuals, and ultimately contribute to enhancing their capabilities in online teaching situations.

Conclusions

The rapid transition to online learning after the onset of the COVID-19 pandemic highlighted the need to provide prospective teachers with essential skills for an online learning environment. This study addresses this need by developing and validating the Pre-service Teacher Competency in Online Learning Environment Scale, using the scenario method to assess pre-service teacher competencies in online settings.

Based on a comprehensive literature review and focus groups, we identified seven pre-service teachers’ competencies. Design/planning competency refers to actions taken before the course to achieve the educational goal. Social competency means teachers’ actions and attitudes to build a rapport with their students in an online learning situation. Instructive competency consists of pedagogical knowledge, content knowledge, and methodological knowledge. In addition, technological competency is the skill required to deliver lectures efficiently and perform various tasks using devices and educational software. Management competency consists of actions necessary for successful teaching and learning experiences in class so that the teachers with management skills can appropriately cope with and solve issues during class. Positive teacher attitude competency is teachers’ job-related attitudes, including teacher self-efficacy and job satisfaction. Lastly, learning competency consists of cognitive/metacognitive sections and student management of different resource sections.

The initial scenario-based scale draft included 13 scenarios and 69 items, which was refined through focus group feedback to a final scale of 11 scenarios and 34 items. The final scale was validated with a survey of 600 pre-service teachers in South Korea, using responses from 579 participants. Reliability was confirmed through Cronbach’s α coefficients, which ranged from .55 to .76 for individual items, with an overall reliability of .93. Content and face validity were ensured by revising or removing items deemed to have low validity through focus groups and surveys. Pearson’s correlation analysis showed strong correlations between the seven competency scores and the total score (.744 to .848), with correlations among the competencies ranging from .539 to .716, confirming both discriminant and convergent validity.

This study acknowledges the unique characteristics of pre-service teachers and identifies competencies tailored to their needs, distinguishing them from in-service teachers. It reflects the evolving demands of education due to the COVID-19 pandemic and outlines the specific competencies required for online teaching. The validated scale provides insights to help teacher training institutions assess pre-service teachers’ skills, ultimately better preparing them for the teaching profession.

Implication

This study has several implications. First, it acknowledges the distinguished characteristics of pre-service teachers and presents competencies tailored to their needs, which differ from those of in-service teachers. Even though pre-service teachers should have teacher and learner competencies as they are in training before entering the teaching field, previous studies did not consider pre-service teachers’ distinguishing characteristics but instead used in-service teachers’ competencies in measuring pre-service teachers (Eren et al., 2015; Ersin et al., 2020; Tang et al., 2016; Yusuf et al., 2017). However, this study confirmed the difference in competencies required by in-service and pre-service teachers and presented competencies for pre-service teachers.

Second, this study is significant in reflecting the educational field’s changes. Due to the spread of online education during the COVID-19 pandemic, there are characteristics in online learning environments distinguished from face-to-face learning environments so that the required competencies of teachers are also different (Alea et al., 2020; Fauzi & Khusuma, 2020). Our research is a timely study exploring pre-service teachers’ competencies required in an online learning environment, which will become more common with technology development.

Lastly, the scale that measures pre-service teachers’ competencies in online learning environments can provide insights for teacher training institutions. This scale suggests realistic scenarios for pre-service teachers to experience and help them diagnose their level before entering teaching and supplement their deficiencies to enhance the quality of pre-service teachers’ training.

Limitation

This study only validated the scale with pre-service teachers in South Korea. Further research might determine if the scale applies to pre-service teachers in different cultural contexts. Additionally, as this study did not differentiate between primary and secondary school levels, it did not examine potential differences in competencies required by pre-service teachers at each level.

As a future direction, we suggest the following enhancements for further research. First, the scale developed in this study should be validated across diverse cultural contexts other than South Korea in further studies. Cross-cultural validation will show if this scale maintains its reliability and validity across different cultural and educational settings, hence giving a wider view regarding the generalizability.

Moreover, subsequent research should separate the school levels into primary and secondary and explore possible differences in the competencies desired by pre-service teachers between the two levels. This would be of value for stage-specific development of competencies and the design of training programs.

Finally, further studies may expand the validation to in-service teachers to compare their competencies with those of pre-service teachers. Comparative studies will shed light on the possible gap or overlap between the two groups and promote continuous improvement in teacher education programs at different professional stages.

Footnotes

Appendices

Appendix B. Descriptive Statistics and Correlations of 34 Items

Item	1-1	1-2	1-3	1-4	2-1	2-2	2-3	3-1	3-2	3-3	4-1	4-2	4-3	5-1	5-2	6-1	6-2	6-3	6-4	7-1	7-2	7-3	8-1	8-2	8-3	9-1	9-2	9-3	10-1	10-2	10-3	11-1	11-2	11-3
Correlations
1-1
1-2	0.292
1-3	0.382	0.254
1-4	0.227	0.334	0.235
2-1	0.340	0.347	0.263	0.202
2-2	0.369	0.300	0.307	0.232	0.319
2-3	0.236	0.240	0.316	0.213	0.366	0.328
3-1	0.367	0.280	0.344	0.256	0.354	0.325	0.436
3-2	0.361	0.229	0.309	0.248	0.322	0.399	0.447	0.553
3-3	0.224	0.176	0.253	0.290	0.181	0.176	0.402	0.356	0.511
4-1	0.334	0.227	0.253	0.207	0.189	0.222	0.153	0.329	0.205	0.108
4-2	0.359	0.301	0.282	0.200	0.380	0.391	0.438	0.356	0.493	0.242	0.274
4-3	0.233	0.327	0.259	0.244	0.278	0.299	0.303	0.272	0.330	0.279	0.238	0.388
5-1	0.330	0.312	0.281	0.234	0.379	0.318	0.391	0.422	0.362	0.304	0.248	0.331	0.345
5-2	0.362	0.207	0.310	0.217	0.314	0.271	0.333	0.409	0.417	0.285	0.280	0.360	0.197	0.446
6-1	0.431	0.353	0.405	0.256	0.432	0.363	0.400	0.454	0.409	0.306	0.289	0.442	0.320	0.376	0.349
6-2	0.296	0.297	0.266	0.191	0.289	0.306	0.277	0.255	0.334	0.275	0.210	0.297	0.163	0.231	0.297	0.402
6-3	0.379	0.242	0.289	0.286	0.278	0.278	0.319	0.376	0.448	0.337	0.184	0.409	0.285	0.297	0.388	0.421	0.391
6-4	0.253	0.206	0.190	0.160	0.259	0.231	0.245	0.184	0.177	0.183	0.071	0.251	0.185	0.191	0.174	0.319	0.267	0.389
7-1	0.381	0.351	0.297	0.272	0.363	0.298	0.357	0.309	0.285	0.186	0.216	0.346	0.199	0.418	0.282	0.409	0.292	0.318	0.281
7-2	0.282	0.276	0.207	0.192	0.282	0.306	0.315	0.330	0.334	0.218	0.263	0.287	0.221	0.264	0.356	0.274	0.268	0.248	0.175	0.457
7-3	0.301	0.236	0.237	0.210	0.362	0.267	0.336	0.324	0.297	0.258	0.158	0.302	0.196	0.258	0.298	0.355	0.244	0.340	0.276	0.342	0.401
8-1	0.433	0.333	0.324	0.272	0.426	0.388	0.457	0.507	0.463	0.280	0.315	0.468	0.334	0.518	0.390	0.542	0.329	0.422	0.269	0.505	0.313	0.321
8-2	0.300	0.295	0.231	0.233	0.247	0.206	0.145	0.295	0.191	0.091	0.295	0.230	0.248	0.225	0.209	0.293	0.259	0.271	0.261	0.290	0.262	0.241	0.433
8-3	0.310	0.320	0.299	0.307	0.251	0.252	0.189	0.298	0.194	0.160	0.249	0.250	0.240	0.233	0.246	0.260	0.226	0.321	0.277	0.293	0.266	0.256	0.395	0.476
9-1	0.272	0.265	0.169	0.147	0.314	0.224	0.345	0.344	0.332	0.248	0.106	0.367	0.215	0.348	0.264	0.471	0.229	0.283	0.204	0.320	0.169	0.342	0.449	0.161	0.141
9-2	0.250	0.266	0.209	0.275	0.241	0.266	0.235	0.271	0.294	0.201	0.254	0.254	0.174	0.258	0.184	0.283	0.206	0.251	0.206	0.269	0.239	0.184	0.346	0.239	0.292	0.232
9-3	0.290	0.141	0.236	0.188	0.238	0.283	0.304	0.298	0.350	0.302	0.222	0.301	0.259	0.247	0.294	0.316	0.287	0.373	0.256	0.207	0.282	0.334	0.345	0.261	0.281	0.243	0.360
10-1	0.405	0.318	0.328	0.232	0.409	0.388	0.367	0.440	0.460	0.299	0.256	0.388	0.339	0.431	0.384	0.493	0.289	0.351	0.238	0.306	0.242	0.346	0.546	0.238	0.266	0.447	0.244	0.304
10-2	0.377	0.154	0.316	0.199	0.328	0.287	0.360	0.403	0.430	0.296	0.224	0.385	0.231	0.324	0.402	0.450	0.256	0.364	0.220	0.298	0.230	0.306	0.428	0.216	0.205	0.367	0.257	0.387	0.567
10-3	0.246	0.216	0.222	0.211	0.245	0.208	0.289	0.184	0.200	0.182	0.132	0.212	0.287	0.188	0.158	0.311	0.190	0.267	0.225	0.219	0.170	0.153	0.251	0.178	0.282	0.213	0.173	0.185	0.330	0.269
11-1	0.301	0.217	0.268	0.151	0.231	0.230	0.229	0.237	0.195	0.121	0.254	0.255	0.238	0.260	0.289	0.341	0.175	0.272	0.239	0.247	0.236	0.203	0.354	0.208	0.226	0.282	0.194	0.179	0.341	0.270	0.218
11-2	0.261	0.255	0.367	0.199	0.306	0.264	0.279	0.201	0.137	0.139	0.170	0.264	0.258	0.290	0.206	0.311	0.208	0.248	0.176	0.294	0.178	0.219	0.314	0.234	0.296	0.144	0.254	0.203	0.215	0.231	0.212	0.209
11-3	0.335	0.268	0.331	0.240	0.306	0.314	0.239	0.282	0.240	0.211	0.254	0.256	0.279	0.247	0.219	0.322	0.239	0.272	0.194	0.266	0.273	0.303	0.278	0.306	0.378	0.187	0.323	0.283	0.328	0.312	0.281	0.186	0.276
Mean	4.050	4.028	4.195	4.081	4.085	4.136	4.294	4.237	4.390	4.314	3.839	4.245	4.235	4.010	4.231	4.219	4.166	4.275	4.090	3.929	4.145	4.282	4.114	4.060	4.124	4.207	3.969	4.214	4.306	4.377	4.010	4.021	4.002	4.073
SD	0.808	0.748	0.792	0.752	0.810	0.800	0.822	0.853	0.776	0.763	0.882	0.799	0.857	0.939	0.795	0.777	0.798	0.741	0.814	0.838	0.817	0.710	0.869	0.822	0.825	0.835	0.864	0.795	0.867	0.778	0.834	0.943	0.843	0.869

ORCID iDs

Jiwon Baek

Jiwoo Kim

Hyeonseong Lee

Youn-Jeng Choi

Ethical consideration

All participants in this study provided informed consent, and data usage has been approved in accordance with institutional ethical guidelines. The research has been reviewed and approved by Ewha Womans University Institutional Review Board (ewha-202205-0024-03), ensuring compliance with relevant ethical standards.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

Due to the privacy of the participants and ethical concerns, the data used in this study are not publicly available.

References

Abdous

(2011). A process-oriented framework for acquiring online teaching competencies. Journal of Computing in Higher Education, 23, 60–77. https://doi.org/10.1007/s12528-010-9040-5.

Alea

Fabrea

Roldan

Farooqi

(2020). Teachers’ COVID-19 awareness, distance learning education experiences and perceptions towards institutional readiness and challenges. International Journal of Learning, Teaching and Educational Research, 19(6), 127–144. https://doi.org/10.26803/ijlter.19.6.8

Ally

(2019). Competency profile of the digital and online teacher in future education. International Review of Research in Open and Distributed Learning, 20(2). https://doi.org/10.19173/irrodl.v20i2.4206

Amrein-Beardsley

(2020). Review of enhancing teacher education, development, and evaluation: Lessons learned from educational reform. Education Review, 27. https://doi.org/10.14507/er.v27.2913

Baek

Kim

(2020). Development and validation of future teacher competency diagnostic scale for pre-service teachers. Journal of the Korea Convergence Society, 11(2), 331–339. https://doi.org/10.15207/JKCS.2020.11.2.331

Bagozzi

(1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Sciences, 16, 74–94. https://doi.org/10.1007/BF02723327

Banerjee

Dolado

Galbraith

Hendry

(1993). Co-integration, error correction, and the econometric analysis of non-stationary data. Oxford University Press.

Begum

Khan

S.A.

(2015). To study the relationship between positive teaching attitude and emotional intelligence of B.Ed. trainees in Aurangabad City. Journal of Educational Psychology, 8, 43–48. https://doi.org/10.26634/jpsy.8.3.3104.

Bhagat

K. K.

L. Y.

Chang

C. Y.

(2016). Development and validation of the perception of students towards online learning (POSTOL). Journal of Educational Technology & Society, 19(1), 350–359.

10.

Bigatel

Ragan

Kennan

May

Redmond

(2012). The identification of competencies for online teaching success. Journal of Asynchronous Learning Network, 16(1), 59–78. https://doi.org/10.24059/olj.v16i1.215.

11.

Byrne

B. M.

(1989). A primer of LISREL: Basic applications and programming for confirmatory factor analytic models. Springer-Verlag Publishing.

12.

Carey

(1994) The group effect in focus groups: planning, implementing, and interpreting focus group research. In Morse

(Ed.), Critical issues in qualitative research methods (pp. 225–241). Sage Publications.

13.

Carmines

E. G.

McIver

J. P.

(1981). Analyzing models with unobserved variables: Analysis of covariance structures. In Bohrnstedt

G. W.

Borgatta

E. F.

(Eds.), Social measurement: Current issues (pp. 65–115). Sage Publications, Inc.

14.

Choi

Lee

Jang

Kim

(2009). Exploring elementary teachers’ core competencies. The Korea Educational Review, 15(3), 103–130.

15.

Comrey

A. L.

Lee

H. A.

(1992). First course in factor analysis. Lawrence Erlbaum Associates, Inc.

16.

Deane

Song

van Rijn

O’Reilly

Fowles

Bennett

Sabatini

Zhang

(2019). The cases for scenario-based assessment of written argumentation. Reading and Writing, 32, 1575–1606. https://doi.org/10.1007/s11145-018-9852-7

17.

Eren

Avci

Z. Y.

Kapucu

M. S.

(2015). Pre-service teachers’ competencies and perceptions of necessity about practical tools for content development. International Journal of Instruction, 8(1), 91–104. https://doi.org/10.12973/iji.2015.817a.

18.

Ersin

Atay

Mede

(2020). Boosting preservice teachers’ competence and online teaching readiness through E-Practicum during the COVID-19 outbreak. International Journal of TESOL Studies, 2(2), 112–124. https://doi.org/10.46451/ijts.2020.09.09

19.

Examsoft. (2021, March 4). How scenario-based assessment contributes to improved learning environments. Retrieved June 8, 2023, from https://examsoft.com/resources/scenario-based-assessment-improves-learning-environments

20.

Fauzi

Khusuma

(2020). Teachers’ elementary school in online learning of Covid-19 pandemic condition. Jurnal Iqra’: Kaijan Ilmu Pendidikan, 5(1), 58–70. https://doi.org/10.25217/ji.v5i1.914

21.

Fehring

R. J.

(1987). Methods to validate nursing diagnoses. Heart & Lung : The Journal of Critical Care, 16(6 Pt 1), 625–629.

22.

Gläser–Zikuda

Fuß

(2008). Impact of teacher competencies on student emotions: A multi-method approach. International Journal of Educational Research, 47, 136–147. https://doi.org/10.1016/j.ijer.2007.11.013

23.

Godber

Atkins

(2021). COVID-19 Impacts on teaching and learning: A collaborative autoethnography by two higher education lecturers. Frontiers in Education, 6(July), 1–14. https://doi.org/10.3389/feduc.2021.647524

24.

Goodyear

Salmon

Spector

J. M.

Steeples

Tickner

(2001). Competencies for online teaching: A special report. Educational Technology Research and Development, 49, 65–72. https://doi.org/10.1007/BF02504508

25.

Greenbaum

T. L.

(1988). The practical handbook and guide to focus group research. Lexington.

26.

Guasch

Alvarez

Espasa

(2010). University teacher competencies in a virtual teaching/learning environment: Analysis of a teacher training experience. Teaching and Teacher Education, 26(2), 199–206. https://doi.org/10.1016/j.tate.2009.02.018

27.

Guest

Namey

McKenna

(2016). How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods, 29(1), 3–22. https://doi.org/10.1177/1525822X16639015

28.

Hair

J. F.

Jr. Anderson

R. E.

Tatham

R. L.

Black

W. C.

(1995). Multivariate data analysis (3rd ed). Macmillan.

29.

Han

Nam

(2020). Faculty competency factor needs analysis to improve the quality of online classes for Higher education. Korean Association for Learner-centered Curriculum and Instruction, 20(13), 1129–1149. https://doi.org/10.22251/jlcci.2020.20.13.1129

30.

Hong

J. Y.

(2010). Pre-service and beginning teachers’ professional identity and its relation to dropping out of the profession. Teaching and Teacher Education, 26(8), 1530–1543. https://doi.org/10.1016/j.tate.2010.06.003

31.

Howard

Hubelbank

Moore

(1989). Employer evaluation of graduates: Use of the focus group. Nurse Educator, 14(5), 38–41. https://doi.org/10.1097/00006223-198909000-00019

32.

Huda

Maseleno

Shahrill

Jasmi

K. A.

Mustari

Basiron

(2017). Exploring adaptive teaching competencies in big data era. International Journal of Emerging Technologies in Learning (iJET), 12(3), 68–83. https://doi.org/10.3991/ijet.v12i03.6434

33.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

34.

Hung

M. L.

Chou

Chen

C. H.

Own

Z. Y.

(2010). Learner readiness for online learning: Scale development and student perceptions. Computers & Education, 55(3), 1080–1090.

35.

Jackson

Gillaspy

Purc-Stephenson

(2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14. https://doi.org/10.1037/a0014694

36.

Jeong

(2020). The concept map on infection disaster response capacity perceived by elementary school teachers: Focusing on Corona19 in Daegu. The Journal of Korean Education, 47(3), 5–31. https://doi.org/10.22804/jke.2020.47.3.001

37.

Kabilan

M. K.

(2004). Online professional development: a literature analysis of teacher competency. Journal of Computing in Teacher Education, 21(2), 51–57. https://doi.org/10.1080/10402454.2004.10784511

38.

Kartal

Başol

H. Ç

. (2019). Generic teacher competencies and the English language teacher education program in Turkey. Novitas-Royal (Research on the Youth and Language), 13(2), 133–154.

39.

Kim

M. M.

Andrews

R. L.

Carr

D. L.

(2004). Traditional versus integrated pre-service teacher education curriculum. Journal of Teacher Education, 55(4), 341–356. https://doi.org/10.1177/0022487104266778

40.

Kim

U. J.

(2009). An examination of the suitability of the view of teaching profession for future education. The Journal of Korean Teacher Education, 26(4), 79–94. https://doi.org/10.24211/tjkte.2009.26.4.79

41.

Kitzinger

. (1995). Qualitative Research: Introducing focus groups. British Medical Journal, 311, 299–302. https://doi.org/10.1136/bmj.311.7000.299

42.

Koc

Barut

(2016). Development and validation of New Media Literacy Scale (NMLS) for university students. Computers in Human Behavior, 63, 834–843.

43.

König

Jäger-Biela

Glutsch

(2020). Adapting to online teaching during COVID-19 school closure: teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education, 43(4), 608–622. https://doi.org/10.1080/02619768.2020.1809650

44.

Krueger

(1994) Focus Groups: A practical guide for applied research (2nd ed.). Sage Publications.

45.

Lee

Kim

Koh

Park

(2011). Development of learning competence test for college students: Focusing on cognition, learning motivation and learning behavior. Korean Journal of Educational Psychology, 25(4), 791–809.

46.

Lin

L. C.

Lin

(2015). Online tutoring service for enriching college students’ experience in community engagement. Procedia–Social and Behavioral Sciences, 191, 1676–1681. https://doi.org/10.1016/j.sbspro.2015.04.691

47.

Martin

Ritzhaupt

Kumar

Budhrani

(2019). Award-winning faculty online teaching practices: Course design, assessment and evaluation, and facilitation. The Internet and Higher Education, 42, 1096–7516. https://doi.org/10.1016/j.iheduc.2019.04.001

48.

Meeus

Van Petegem

Engels

(2009). Validity and reliability of portfolio assessment in pre-service teacher education. Assessment & Evaluation in Higher Education, 34(4), 401–413. https://doi.org/10.1080/02602930802062659

49.

Ministry of Education (South Korea) and Korean Educational Development Institute (2022). Education statistics compilation - kindergarten, elementary, middle, and high school education statistics. Korean Educational Development Institute.

50.

Morgan

(1996). Focus groups. Annual Review of Sociology, 22, 129–152. https://doi.org/10.1146/annurev.soc.22.1.129

51.

Muthuprasad

Aiswarya

K.S. Aditya

K. S.

Girish

K. J.

(2021), Students’ perception and preference for online education in India during COVID -19 pandemic, Social Sciences & Humanities Open, 3(1). https://doi.org/10.1016/j.ssaho.2020.100101

52.

OECD (2009). Teaching Practices, Teachers’ Beliefs and Attitudes. In Creating Effective Teaching and Learning Environments: First Results from Teaching and Learning International Survey (TALIS) (pp. 87–135). OECD Publishing. https://doi.org/10.1787/9789264068780-6-en

53.

OECD (2019), Supporting and guiding novice teachers: Evidence from TALIS 2018. Teaching in Focus, 29, 1–6. https://doi.org/10.1787/23039280

54.

Y. B.

(2020). Exploring teacher’s distance instruction competencies through non face-to-face distance instruction case analysis: Based on synchronous distance instruction. Journal of Educational Technology, 36(3), 715–744. http://doi.org/10.17232/KSET.36.3.715

55.

Pillay

Irving

Tones

(2007). Validation of the diagnostic tool for assessing tertiary students’ readiness for online learning. High Education Research & Development, 26(2), 217–234.

56.

Pintrich

Smith

Garcia

McKeachie

(1991). A Manual for the use of the motivated strategies for learning questionnaire (MSLQ). https://doi.org/10.1037/t09161-000

57.

Pokhrel

Chhetri

(2021). A literature review on impact of COVID-19 pandemic on teaching and learning. Higher Education for the Future, 8(1), 133–141. https://doi.org/10.1177/2347631120983481

58.

Rabiee

(2004). Focus-group interview and data analysis. Proceedings of the Nutrition Society, 63(4), 655–660. https://doi.org/10.1079/PNS2004399

59.

Reyes-Fournier

Cumella

March

Pedersen

Blackman

(2020). Development and validation of the Purdue global online teaching effectiveness scale. Online Learning. 24. https://doi.org/10.24059/olj.v24i2.2071

60.

Rocha

F, W.

Holm

Svensson

Ericsson

. (2014). Using phishing experiments and scenario-based surveys to understand security behaviours in practice. Information Management & Computer Security, 22(4), 393–406. https://doi.org/10.1108/IMCS-11-2013-0083

61.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

62.

Sahito

Väisänen

(2017). Effect of time management on the job satisfaction and motivation of teacher educators: a narrative analysis. International Journal of Higher Education, 6(2), 213–224. https://doi.org/10.5430/ijhe.v6n2p213

63.

Seferoglu

S. S.

(2005). A study on teaching competencies of teacher candidates. Proceedings of International Conference on Education. National University of Singapore.

64.

Selvi

(2010). Teachers’ competencies. Cultura International Journal of Philosophy of Culture and Axiology, 7(1), 167–175. https://doi.org/10.5840/cultura20107133

65.

Shulman

L. S.

(1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X01500200

66.

Sokal

Trudel

L. E.

Babb

(2020). Canadian teachers’ attitudes toward change, efficacy, and burnout during the COVID-19 pandemic. International Journal of Educational Research Open, 1(2020). https://doi.org/10.1016/j.ijedro.2020.100016

67.

Spencer

L. M.

Spencer

S. M.

(1993). Competency at Work: Models for superior performance. John Wiley & Sons.

68.

Stewart

Shamdasani

(1990). Focus groups Theory and practice. Sage Publications.

69.

Tabachnick

Fidell

(2007). Using multivariate statistics. (5th ed). Pearson Education Inc.

70.

Taber

K. S.

(2018). The use of Cronbach’s Alpha when developing and reporting research instruments in science education. Research in Science Education, 48, 1273–1296. https://doi.org/10.1007/s11165-016-9602-2

71.

Tang

S. Y. F.

Cheng

M. M. H.

Wong

A. K. Y.

(2016). The preparation of pre-service student teachers’ competence to work in schools. Journal of Education for Teaching, 42(2), 149–162. https://doi.org/10.1080/02607476.2016.1143143

72.

Tang

(2022). The impact of teacher attitude and teaching approaches on student demotivation: Disappointment as a mediator. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.985859

73.

Thompson

Daniel

(1996) Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56, 197–208. https://doi.org/10.1177/0013164496056002001

74.

Uzunboylu

Ozdamli

(2011). Teacher perception for m-learning: scale development and teachers’ perceptions. Journal of Computer assisted learning, 27(6), 544–556. http://doi.org/10.1111/j.1365-2729.2011.00415.x

75.

Welch

Napoleon

Hill

Roumell

(2014). Virtual Teaching Dispositions Scale (VTDS): A Multi-dimensional instrument to assess teaching dispositions in virtual classrooms. MERLOT Journal of Online Learning and Teaching, 10(3), 446–467.

76.

Yoon

(2021). Study on the instructors’ competence of digital literacy for online classes in the untact era. The Journal of Humanities and Social Sciences 21, 12(2), 2625–2638.

77.

Yusuf

F. N.

Widiati

Sulistyo

(2017). Multimodal feedback provision in improving pre-service teachers’ competence. Indonesian Journal of Applied Linguistics, 7(2), 239–246. https://doi.org/10.17509/ijal.v7i2.8126

78.

Zacharis

G. K.

(2020). Investigating the factors influencing pre-service teachers’ acceptance to use mobile devices for learning: The case of a Greek university. In Papadakis

Kalogiannakis

(Eds.), Mobile learning applications in early childhood education (pp. 183–208). IGI Global. https://doi.org/10.4018/978-1-7998-1486-3.ch010

79.

Zhumash

Nurgaliyeva

Zhumabaeva

Lebedeva

Saduakas

Zhoraeva

(2021). Professional teaching competence in preservice primary school teachers: Structure, criteria and levels. World Journal on Educational Technology Current Issues, 13, 261–271. https://doi.org/10.18844/wjet.v13i2.5699