Abstract
This study intends to investigate the factors which can affect the sustainability of mobile learning (m-learning) from many perspectives. Current studies on the sustainability of m-learning are reviewed in the context of organizational, financial, legal and ethical, pedagogical, assessment, psychological, and social factors. In addition to the literature review, an investigation based on formal interviews is conducted with 11 heads in university centers for distance education (UCDE) to obtain additional sustainability factors. To test the impacts of the factors on sustainability in m-learning, another investigation which bases on survey research is conducted with administrative personnel in UCDE, and instructors and students in the distance education universities. A total of 667 survey responses are analyzed, and a model for the sustainability of m-learning is developed. This article concludes that some factors have significant effects on sustainability in m-learning and some of them are critically important for m-learning initiatives. M-learning sustainability is studied rarely in some contexts such as organizational, psychological, and social context. Therefore, this study can bridge this gap, and provide a guideline to m-learning initiatives for sustainable education.
Keywords
Introduction
Mobile devices are used not only in personal life and working life but also in education as part of mobile learning (m-learning). Tatar et al. (2003, p. 30) defined m-learning as “the intersection of online learning and mobile computing.” Learning in a mobile environment is defined as “any form of learning delivered to students anytime and anywhere through mobile devices” by Wang et al. (2009). Some of the mobile devices used in m-learning can be listed as cellular phones, smartphones, personal digital assistants (PDAs), tablets, and so on (Andronico et al., 2003). While in 2007, the generation of laptop and tablet PCs cannot be named as mobile devices (Traxler, 2007), currently, some researchers state that laptop and tablet PCs can be also considered as mobile (Kalinić & Arsovski, 2009). University students may use mobile devices as a medium for communication, management of learning materials, effective learning, collaborative learning, and the development of new ideas (M. S. H. Khan et al., 2019). Some mobile device usages in instructional activities are downloading and sharing documents, collaborating on projects, reviewing coursework, preparing for exams, sharing project results, reading (listening to) audio books, recording information, presenting projects, conducting research, saving their work, submitting work to the instructor, and so on (Corbeil & Valdes-Corbeil, 2007).
The importance of online education has been better understood since the spread of the coronavirus disease in 2019 (COVID-19). With the closure of schools in long quarantine days, there has been more need to use of mobile devices in learning. Even if mobile device usage is spreading among educational institutions, as Goundar (2011, p. 1) stated, “many stakeholders in the education system are already struggling to deliver basic education.” The high dropout rate in distance learning is a problem for many organizations (J.-H. Park, 2007; Sulčič & Sulčič, 2007) and failures exist in many distance learning initiatives (Stepanyan et al., 2010; Sun et al., 2008).
Wingkvist (2009) argues that complex issues may lead to adoption problems in m-learning initiatives, and the initiatives may not succeed to sustain. Adoption is not enough for sustainable m-learning. The initiatives should also meet the educational intents, deal with difficulties, make progress or maintain a certain condition, and be prepared for changes (Coskun-Setirek & Tanrikulu, 2015).
The term “sustainability” can be interpreted from many different viewpoints. Sustainability was defined by Eckersley (1998, p. 6) as “the ability to continue an activity or a certain condition indefinitely”. Sustainability of education focuses on the implementation of sustainable forms of “successful” practice through educational development, leadership, and innovation (Davies & West-Burnham, 2003). Robertson (2008, p. 819) interpreted sustainable e-learning as “e-learning that has become normative in meeting the needs of the present and future.” Sustainability is “the property of e-learning practice that evidently addresses current educational needs and accommodates continuous adaptation to change, without outrunning its resource base or receding in effectiveness” (Stepanyan et al., 2010, p. 95). In Gunn’s (2010, p. 90) study, an e-learning initiative is considered sustainable when one of the conditions for sustainable e-learning is that “the e-learning concept, design, system or resources have proven potential to be adopted, and possibly adapted.” The main difference between e-learning and m-learning is in the technologies used for educational content supply according to Georgiev et al. (2006) while in the addition of capabilities and limitations in the evolution aspects according to Alzaza and Yaakub (2011).
Based on these definitions, the following abilities can be referred for the sustainability of m-learning: the ability to address current educational needs and intent of m-learning, ability to have potential to be adopted by users, ability to maintain a certain condition indefinitely or make progress, and ability to adapt to possible changes. Universities and other m-learning initiatives must meet the increasing demand for m-learning with sustainable education policy. This study intends to investigate the organizational, financial, legal and ethical, pedagogical, assessment, psychological, and social factors that can affect the sustainability of m-learning. Especially organizational, psychological, and social perspectives of m-learning sustainability are studied rarely. Therefore, this study can bridge the gap, and provide a guideline to m-learning initiatives for sustainable education.
This study tries to answer the following research question:
Theoretical Background
The two broad directions of sustainability in education are education for sustainability and sustainability of education. Education for sustainability focuses on sustainability through educational solutions, whereas the sustainability of education focuses on the implementation of sustainable education through development, leadership, and innovation (Stepanyan et al., 2013). In this study, since we have focused on the sustainability of m-learning, the literature on “sustainability in higher education focused on m-learning” was reviewed instead of “higher education for sustainable development.” The sustainability in higher education has been studied from a variety of perspectives, for example, researchers studied on a sustainable university model (Velazquez et al., 2006), sustainability in higher education through interaction with lecturers (Holmberg et al., 2008), integration of sustainable development in curricula of higher education (Ceulemans & De Prins, 2010), and sustainable consumption in universities (Barth et al., 2014).
There are few studies that investigated m-learning sustainability in some contexts such as organizational, psychological, social, and so on, while the technologic perspectives of sustainability in m-learning are widely studied. However, there are some studies which can be base for such contexts. In the light of sustainability abilities, when the sustainability items are studied, educational needs, adoption, success factors, limitations and challenges, and changes and risks of m-learning projects are also included in this study, considering the difference between e-learning and m-learning. The sustainability of m-learning is examined under related aspects.
Organizational Aspects
Researchers examined m-learning from an organizational perspective. Wang et al. (2007) inspected m-learning in an organizational context with such issues: system, information, and service quality; system use; user satisfaction; and net benefits as individual impact and organizational impact. Stansfield et al. (2009) specified some organizational aspects: teamwork, roles, and responsibilities; bureaucracy and administration; language, culture, and gender; and project management and leadership. Conole (2004) examined the following issues: users’ roles and responsibilities, structures and processes, and context and culture. He specified also some important consolidation aspects which are involving stakeholders, continued evaluation, dissemination, and planning for sustainability. Coordination, negotiation, and collaboration are some factors for m-learning adoption (Zurita & Nussbaum, 2004). Kong (2019) identified the partnership structures and elements for sustainable e-learning development. Ng and Nicholas (2013) stated that inclusive and communicative leadership is important for sustainable m-learning. Clear policy and guidelines needed, interdisciplinary collaborations for development, and relevant infrastructure and support are three organizational requirements for sustainability in m-learning initiatives (Murphy et al., 2016).
Two of the principles which are emphasized by Chipere (2017) are stakeholder-centeredness and high operational efficiency for sustainability in e-learning programs. Ahmad et al. (2018) specified some managerial success factors such as organizational infrastructure readiness and training of stakeholders. Stepanyan et al. (2013) studied some organizational factors like teacher training and development, institutional transformation, and educational leadership under the domain of professional development and innovation. According to Hoa et al. (2015), an e-learning environment requires institutional support from the government for sustainability. Training of users and support for faculty are critical success factors in e-learning systems (Naveed et al., 2020). Finding and retaining staff, getting employees to make time, and lack of management involvement are some important organizational challenges for e-learning initiatives (Berge & Kearsley, 2003).
Financial Aspects
From a financial viewpoint, some studies have examined the factors which can affect sustainability in m-learning. To increase the quality of education, financial funding consideration is important (Nenadić et al., 2012). For example, an increase in investments for hardware and internet connections is required in m-learning projects (Casany et al., 2012). Stansfield et al. (2009) specified some financial aspects for sustainable e-learning initiatives: appropriate costing methods, cost–benefit analysis, and wider institutional context. Stepanyan et al. (2013) also studied some financial aspects like cost-effectiveness, cost-efficiency, and economies of scale and scope under the domain of resource management. Cost-effectiveness is one of the principles which are emphasized for e-learning sustainability (Bates & Poole, 2003; Chipere, 2017). According to Traxler (2007), costs for content development, teaching, software development, hardware, usage, and phone charges should be considered separately in m-learning projects. Timpone (2012) indicated such costs: instructional and administrative tools, infrastructure, system development and maintenance, support, and connectivity costs. The cost of a transaction is an important limitation for m-learning initiatives (Alzaza & Yaakub, 2011). Upper management wants a demonstration return on investment (ROI) and a reduction in training cost, and these provide cost-effectiveness and cost-efficiency (Raths, 2001). According to Hoa et al. (2015), an e-learning environment requires financial support from the government for sustainability.
One of the most common barriers or issues to e-learning implementation is the lack of financial resources (Clarke et al., 2005). Successfully orchestrating a multicampus faculty development program requires a good deal of resources, both monetary and human (Reilly et al., 2012). Budget limitations, system development costs, and demonstrating ROI for e-learning are some of the financial challenges of e-learning initiatives (Berge & Kearsley, 2003). According to Reilly et al. (2012), the primary precondition is to make a good and well worked out financial plan at the institution level.
Legal and Ethical Aspects
Some legal and ethical issues are studied in the literature. As ethical and legal issues, privacy and personal data protection, and security are studied by many researchers (Dolawattha et al., 2019; Naveed et al., 2020; Wang & Heffernan, 2010). Mason (2017) indicated some ethical concerns in the information era: privacy, accuracy, ownership, and accessibility. Wright (2011) studied on ethical and safe information and communication technology practices, and listed the following legal, ethical, and safety factors:
Legal: Copyright infringement/appropriate attribution; discrimination: disability, racial, religious; hacking, breach of terms and conditions; theft; defamation: libelous content; and sedition.
Ethical: Plagiarism/fraud; netiquette; cultural sensitivity, blasphemy, obscenity; intimidation, harassment, stalking, blackmail; image manipulation; and respect.
Safe: Privacy issues; security of information: passwords; financial details; stranger danger—online predators; cyberbullying; exposure to inappropriate content; and health and safety due to overuse.
In Global System for Mobile Communications (2012) Connected Living Programmed: m-Education, potential risks are handled under the title safeguarding, security, and privacy. Accessing inappropriate content and communication, mobile bullying, distraction, financial difficulty, and illegal file-sharing/downloading are given under safeguarding. According to them, learner privacy and autonomy, and data and system privacy and security are other risks of mobile education.
Ahmad et al. (2018) specified some managerial success factors such as security, access control and privileges, and commitment to enhancing the sustainability in e-learning. According to Wishart (2009), commonly accepted ethical principles can be applied to the six areas which are personal information, images, informed consent, ownership, data storage and protection, and user-generated content. Lally et al. (2012) examined the ethical aspects, and the following factors are revealed: informed consent, access to technology, user-generated content, attachment, introducing unsuitable materials, the intrusion of privacy, and unmonitored spaces. Social and political influence, cultural diversity, bias, geographical diversity, learner diversity, information accessibility, etiquette, and legal issues are related to ethical issues of e-learning (B. H. Khan, 2001). Dyson et al. (2017) studied the following ethical issues: ensuring equality of access; distraction, disruption, and time-wasting; banning or restricting access; resource wastage; discouragement to future participation; privacy and sharing personal information; and cheating.
According to Alzaza and Yaakub (2011), concerns over confidentiality of personal information are important limitations. Traxler and Bridges (2005) inspected the following ethical and legal challenges of m-learning: informed consent, participant risk, participant withdrawal, payment or compensation, confidentiality and anonymity, private and public distinctions, roles, status and power, cultural differences, and developing effective debriefing and netiquette.
Ally and Tsinakos (2014) stated that it is required to guard against possible risks. They studied on increasing access through m-learning and specified five top risks of m-learning as entrenched digital divides: digital inequality, digital literacy, and digital citizenship; digital distractions and threats; the opposite of a green manifesto; uncontrolled, misleading effects on outcomes; and poor ROI.
Pedagogical Aspects
Researchers handled m-learning in terms of pedagogic aspects. A sustainability model for m-learning combines multiple factors, including pedagogy (Ng & Nicholas, 2013). M-learning environments should be supported by comprehensive theoretical foundations (Daughtery & Berge, 2017). Naismith et al. (2004) and Kailasrao (2012) handled the significance of main theories in m-learning. Stansfield et al. (2009) specified some important pedagogical sustainability issues which are appropriate pedagogical models and approaches, appropriate and stimulating content, embracing innovation, quality evaluation, and support for staff and users/students. Granic and Cukusic (2011) studied an educational evaluation of an e-learning platform and they conclude that accurate quality assessment is provided by end-user testing.
Ktoridou and Eteokleous (2005) studied on the following pedagogical m-learning subjects: curriculum and learning materials development, contents, pedagogical methods, and instructional approaches. James (2008) emphasized the learning strategies, models, and applications; virtual networked and collaborative learning practices; staff support; and communication patterns and activities among stakeholders. Meeting educational needs by e-learning is an important pedagogical challenge of e-learning initiatives (Berge & Kearsley, 2003). Personalized adaptive learning and lack of standards are some pedagogical challenges of m-learning (Wolpers & Grohmann, 2005). Attewell and Savill-Smith (2004) examined the changes in teaching and learning in their study under such titles: individualization, collaborative and active learning, informal learning with multiple media, and cognitive and behavioral change. Georgiev et al. (2006) examined the current limitations of the transition to m-learning from a student, educator, and developer perspective. According to them, some pedagogical challenges are little knowledge about different pedagogical approaches and technological limitations of mobile devices, the focus of m-learning, and the requirement for being self-organized.
Mostakhdemin-Hosseini (2009) implicated that the major pedagogical issues in m-learning are “the content of the learning materials” and “the context where the learning occurs.” Conole (2004) collected the research themes and questions under the hood of pedagogical aspects; these are understanding the learning process, design and development issues, delivery and integration, support and evaluation, and quality assurance. According to Stepanyan et al. (2013), educational attainment is one of the sustainable e-learning domains which include the benefits, quality, usability, and student performance themes. Abramson et al. (2015) and Dolawattha et al. (2019) stated that usefulness is a significant factor for m-learning adoption.
There are other studies about the pedagogical aspects of m-learning. Kearney et al. (2012) inspected such pedagogical issues of m-learning: authenticity, collaboration, and personalization. Park (2011) categorized educational applications of mobile technologies with respect to socialized and individualized m-learning. Villaverde et al. (2006) studied the recognition of learning styles in e-learning systems. They state that there are some e-learning systems adapting their behavior to a learner’s style and this is required for effectiveness. Laurillard (2007) studied pedagogical forms for m-learning. The study handled “whether an m-learning design motivates students for some issues or not.”
Assessment Aspects
Assessment is “probably the most important thing we can do to help our students learn” (Brown, 2005). The assessment issues of m-learning systems were also inspected by many researchers. As assessment strategies for online courses, Gaytan and McEwen (2007) lay emphasis on variety and clarity assignments; continual, immediate, and detailed feedback; understanding of assignments; the quality of interaction; and a variety of assessment techniques. Robles and Braathen (2002) studied on online assessment techniques and handled such issues: understanding the assignments, understanding the content material, different learning styles, comparability of the rigor of the online course, student opinions about the course, being sure students participate, group cohesiveness, and meeting the learning outcomes.
Iahad et al. (2004) evaluated the online assessment as usability and functionality evaluation. According to that study, usability depends on the loading and response time of a system, ease-of-use of the system, and being informative of tests, while functionality bases on delivery, assessment, and feedback. Sun et al. (2008) proved that feedback from more or diversified assessment in e-learning system increases learners’ satisfaction. According to Nikou and Economides (2017), the content and feedback positively affect the usefulness of mobile assessment tools, while ubiquity value and user interface affect positively its ease of use. Moreover, the perceived fairness and the perceived soundness positively influence the examinee’s perceived trust of online exams and perceived participation acceptance (Liu et al., 2015). According to Naveed et al. (2020), timely appropriate feedback is a critical success factor in implementing e-learning system. Buchanan (2002) listed some challenges of m-assessment as breakdowns in connectivity, validity, secure server and encrypted communications, scoring, different populations, and practical and ethical issues. Graff (2003) claimed that possessing poor information technology (IT) skills is a drawback of online assessment.
Psychological Aspects
M-learning systems have different learning processes with respect to traditional learning systems. M-learning occurs across times and contexts, and this causes some psychological challenges which are important to understanding for effective m-learning (Terras & Ramsay, 2015). The context-dependent nature of memory, finite human cognitive resources, distributed cognition and situated learning, metacognition, and learner differences matter are essential for m-learning (Terras & Ramsay, 2012). Because of such changes, some issues should be addressed, for instance, overlapping of educational contents and learning styles of learners (Franzoni et al., 2008).
Learners possess psychological infrastructure in an effective m-learning initiative (Terras & Ramsay, 2012). Learning should engage learners in the learning process in addition to good learning outcomes (Shen et al., 2009). The success in m-learning implementation in universities can be related to users’ acceptance of m-learning (Abu-Al-Aish & Love, 2013). They argue that creating normative expectations with an e-learning community and sharing successful e-learning experiences of employees can support the adoption of their e-learning. Chokri (2012) argues that the adoption of e-learning systems in education is important and the sustainability of the systems in education is impossible without understanding the reasons for students’ interest in e-learning.
Sørebø et al. (2009) and Betancourt-Odio et al. (2021) studied the motivation of lecturers for e-learning/m-learning and sustainability. Ciampa (2014) has investigated the student motivation for learning in a mobile age. Wagner et al. (2008) also think that meeting the needs and addressing the issues of key stakeholders are important for the success of e-learning systems. Sfenrianto and Suhartanto (2011) stated that adopting learning styles and motivation improve learning performance. According to Rad and Rezaei (2015), m-learning increases achievement motivation, emotional experiences, self-concept, and student self-confidence. Attitude toward e-learning/m-learning, self-efficacy, and motivation are also studied as critical success factors (Dolawattha et al., 2019; Fırat et al., 2018; Naveed et al., 2020). Abramson et al. (2015) found self-efficacy significantly correlated with the user’s behavioral intention to adopt m-learning.
Social Aspects
Some studies discuss the social environment and interaction in m-learning systems. According to Preece (2000), people, purpose, policies, and environments can be used for establishing social interaction.
Lowenthal (2010b, p. 1) stated that “theories of social presence help explain how students and teachers interact and learn online.” Aragon (2003) laid emphasis on welcome messages, learner profiles, audio content, class size, and collaborative learning activities for the establishment of social presence. Contribution to discussion boards, quick response to e-mail, promptly feedback, conversation, the share of personal stories and experiences, use of humor, use of emotions, addressing learners with their names, and options for addressing the lecturer are some strategies for e-learning instructors according to him. The enrichment of social context awareness, location-based communication management, personalization of multilayered interactivity, and optimization of digital and social identities are essential to enhance the mobile social presence of students and effective interaction (Tu et al., 2016). Social presence has a significant impact on learners’ satisfaction (Alsadoon, 2018; Ooi et al., 2018).
The interaction among learners has a more important impact on the e-learning than learner–instructor interaction (Essam & Al-Ammary, 2013). Moreover, the interaction way of learners and instructors in online courses is essential and related to discourse (Lowenthal, 2010a). The discourse is defined as “purposeful conversation or dialogue” in E-Primer Series (Nichols, 2008). Nichols examined the discourse in terms of the expectations, opportunities, messages, engagement, written style, and teaching presence. Dolawattha et al. (2019) and Naveed et al. (2020) also studied interactivity, communication, and collaboration as critical success factors for e-learning/m-learning.
Method
Research Design
This study includes mixed (both qualitative and quantitative) research methods. With the qualitative research, benefiting from experiences, perspectives and ideas of experts, and reducing restrictions are aimed. With the quantitative research, it is aimed to reach more person concerned and to increase the generalizability and the objectivity of the study.
The research design of this study consists of five phases (Figure 1). In Phase I, the literature is reviewed on the sustainability in m-learning to obtain literature-based sustainability items of m-learning. In Phase II, an investigation is made to obtain interview-based sustainability items of m-learning from experts’ view. In Phase III, the literature-based sustainability items are combined with the interview-based items, and survey questionnaires are developed with these items to conduct with experts. In Phase IV, factor analysis is applied to survey data to categorize the sustainability items under the sustainability factors of m-learning. Afterwards, the hypotheses and the theoretical frameworks for the sustainability of m-learning are developed using these factors. In Phase V, descriptive analysis and regression analyses are applied to understand the impacts of the obtained factors on the sustainability of m-learning and to test the hypotheses. The final framework for the sustainability of m-learning is proposed in this phase.
Research design.
Data Collection and Instrumentation
In this study, EBSCOhost was used as a research platform for data collection from literature. This platform was preferred because it provides a range of library database services and includes various databases like “Academic Search Complete,” which is one of the most comprehensive scholarly, multidisciplinary full-text databases, and “ERIC” database, which is produced by the Education Resources Information Center. Because ERIC is an online library of education research and information, it is one of the most appropriated sources for m-learning study. The platform covers a significant number of journals indexed in Web of Science (WoS) or Scopus.
To find the relevant article, the terms “blended learning,” “online courses,” “distance learning,” “online education,” “distance education,” “e-learning,” “electronic learning,” “m-learning,” or “mobile learning” were searched in abstracts (with the “AB abstract” field code) along with the keyword “sustainab*” in all text of articles (with the “TX All Text” field code) without any time restriction. “Apply equivalent subjects” is selected as an expander. The result was filtered for the peer-reviewed scholarly journals and full-text publications for the review. It is limited also to publications written in English. Academic journals and conference materials were selected as source types. From this initial search, totally 1,977 publications were obtained. For more related articles, EBSCOhost provides subject tags. The platform indexes all these articles. The platform maintains subject terms’ Comprehensive Subject Index (CSI), which is an expansion and adaptation of the Library of Congress Subject Headings database. These subject tags are used for the subject filtering and following subjects are selected: “distance education,” “higher education,” “educational technology,” “mobile learning,” “online education,” “postsecondary education,” “sustainability,” “blended learning,” “internet in education,” and “sustainable development.” After this filtering, totally 747 publications were obtained for the review.
The distribution of articles by databases and journals was displayed. The highest number of articles belonged to the databases “Academic Search Ultimate,” “The Belt and Road Initiative Reference Source,” “Business Source Ultimate,” and “ERIC,” respectively. The highest number of articles that belonged to the journals was also listed.
The abstracts of these articles or full articles were reviewed, and unrelated articles were eliminated by two authors of this study. The elimination criterion was that the article should include sustainability items for higher education organizations, which are implementing e-learning or m-learning. When the sustainability items are searched, educational needs, adoption, success factors, limitations and challenges, and changes and risks of m-learning projects are also searched in articles in the light of the defined abilities for sustainability of m-learning.
At the end, the final number of articles, from which the sustainability items are obtained, was reduced to 77. The remaining articles were reviewed for the purposes of the data analysis. The number of articles per the publication year range is given in Table 1. The publications including the e-learning/m-learning sustainability items are between the years 2000 and 2020. Previously published articles have been eliminated for failing to comply with the criterion.
The Number of Articles per Publication Year Ranges.
The profiles of journals with more than two articles reviewed are given in Table 2. Impact factor, Scimago Journal Rank (SJR), h index, quartile in the area of education, and average citation per document data were obtained from Clarivate’s (2020) Journal Citation Reports (JCR), which is powered by WoS, and Scimago’s (2020) journal ranking (SJR), which is powered by Scopus. These impact indicators measure the importance of a publication within a specific subject area. Table 2 shows that the highly cited journals in this study are in “Quartile 1” or “Quartile 2” in the field of education.
Journal Profiles of the Final Reviewed Articles.
Note. SJR = Scimago Journal Rank; JCR = Journal Citation Reports.
Furthermore, the ideas and opinions of experts were obtained with structured interviews to obtain additional m-learning sustainability factors. “Structured interviews are based on a predetermined and standardized identical set of questions” (Saunders, 2011). The interview questionnaires consisted of six predetermined and standardized questions and a comment area, considering the m-learning sustainability abilities in the literature review part of this study (Appendix A). The study aims and the interview process were explained to the experts. The issue of confidentiality was also addressed, and the contact information of the interviewer was given. The interviews were conducted through e-mail.
For the survey research, seven survey questionnaires were formed. The survey questionnaires start with five descriptive items: these are name (not required), institution (not required), age, position, and working or learning year in an m-learning environment. They contain another item related to accessibility to learning systems from mobile devices. It is a critical survey item because if there is no accessibility from mobile devices, no need to answer other items. The sustainability parts of the seven questionnaires include 73 Likert-type scale survey items for the literature- and interview-based m-learning sustainability items (can be seen in the “Analysis Results” section). Each survey questionnaire includes also a Likert-type scale item (totally seven items) to measure the sustainability of each aspect. Finally, five Likert-type scale items were prepared for the general sustainability of m-learning.
Population and Sampling
Heads in university centers for distance education (UCDE) are the interview population of this study. For the interview investigation, a purposive sampling technique was used to obtain appropriate responses, and 11 interviewees were reached. The interviewee profiles are given in Appendix B. They all have more than 5 years of experience in executive director or professor at an UCDE.
Administrative, system, design, and development staff in UCDE, and instructors and students at distance learning universities are the population of survey research. Snowball sampling was used for survey research. According to Baltar and Brunet (2012), snowball sampling is a useful methodology in studies that respondents are few in number and hard to reach. First, the heads in UCDE were contacted via telephone, and e-mail contacts of other administrative personnel were demanded. Then, the link of the survey form was sent through electronic mail. The survey research includes seven questionnaires, and the related questionnaire was conducted to the related sample (Table 3). Totally 667 completed survey responses have been collected from the administrative, system, and design and development staff, instructors, and students. The sample size of each part is given in Table 3.
Sample Size of the Survey Research.
According to Hair et al. (1995), the minimum ratio of subjects to independent variables is 5:1 for multiple regression analysis, and Miller and Kunce (1973) and Halinski and Feldt (1970) reported 10:1 as a more conservative ratio (as cited in Kotrlik & Higgins, 2001).
Data Analysis
The content analysis technique was used to analyze the literature and interview data. Content analysis “classifies textual material, reducing it to more relevant, manageable bits of data” (Weber, 1990, p. 5). This research method seems appropriate for analyzing qualitative data. The literature and interview data were analyzed to reduce the data to m-learning sustainability items by two authors of this study.
In this study, articles and interview manuscripts were reviewed to identify the sustainability items and categories. Including the recurring and synonyms, a total of 585 sustainability issues, 381 of which were from articles and 204 were from interview manuscripts, are obtained. After the elimination of the recurring issues and aggregating the issues that have a similar meaning, a total of 73 sustainability items, 58 of which were from articles and 15 were from interview manuscripts, were determined by the consensus of researchers.
The researchers, a professor and a research assistant in the Department of Management Information Systems, analyzed the items as two coders. The coders independently aggregated the items into a smaller number of broad categories. Totally 11 basic unit categories were collected from the two coders. These categories were evaluated together by the researchers. It was decided to examine the consolidation issues under the organizational aspect and instructional issues under the pedagogical aspect by reviewing their scope. Moreover, the legal and ethical aspects are combined. As the security themes are related to the assessment, it was decided to examine them under the subject of assessment. At the end of this process, seven final theme categories were specified as m-learning sustainability aspects (Table 4).
Kappa Statistics.
Not assuming the null hypothesis.
Using the asymptotic standard error assuming the null hypothesis.
To determine whether the 73 items could be placed into the same categories by two independent judges, reliability analyses were conducted. They worked independently and assigned the list of themes to one of the seven categories. Each item was placed under a theme category.
Intercoder reliability check was used to express the extent of agreement achieved among independent judges regarding the assignment of the same items to the categories (Mouter & Vonk Noordegraaf, 2012). One way of measuring intercoder reliability is the percentage match. To determine the accuracy of coding, the percentage of agreement between the two judges was used. A crosstab table was created, and the number of items assigned to the same category was calculated to obtain the agreement level between two judges. In this study, the agreement level was found as 85% (Table 4) and that corresponds to the coefficient of agreement recommended by Kassarjian (1977).
Statistical analysis techniques were used to analyze the survey data. The data were, first, transferred into Excel 2013 and then transferred into the data analysis tool Statistical Package for the Social Sciences Version 24 for the descriptive, factor, reliability, and regression analyses.
The Validity of the Study
Content and construct validity are established for this study. Content validity is defined as “the degree to which the content of the items adequately represent the universe of all relevant items under study” (Blumberg et al., 2008). First, face validity (content validity) is established by examining the topic carefully and developing the questionnaire in respect of literature. Then, the “panel of judges” validity (content validity) is established by consulting with experts about the development of the questionnaire. Factorial validity is one form of construct validity (Lu, 2007), and the items are grouped by factor analysis with appropriate values (as presented in the section “Analysis Results”) before hypothesis testing.
Analysis Results
Phase I
In this phase, the literature is reviewed on the sustainability in m-learning. It is examined under the titles organizational, financial, legal and ethical, pedagogical, assessment, psychological, and social aspects. The literature-based sustainability items of m-learning are specified in this phase (Table 5).
Literature-Based Sustainability Items of M-Learning.
Phase II
In Phase II, an investigation is made to obtain additional sustainability items of m-learning from experts’ view. The investigation is based on formal interviews and conducted with heads in UCDE. Interview-based sustainability items of m-learning are obtained in this phase (Table 6).
Interview-Based Sustainability Items of M-Learning.
Phase III
In this phase, the literature-based sustainability items of m-learning are combined with the interview-based items. Seven survey questionnaires are developed with these items and are used for another investigation to obtain data about the sustainability of m-learning. This investigation is based on survey research and conducted with administrative, system, design, and development staff in UCDE, and instructors and students at distance learning universities. The survey data are obtained and prepared for analyses in this phase.
Phase IV
In this phase, factor analysis is applied to survey data to categorize the sustainability items under the sustainability factors of m-learning. The frameworks and the hypotheses for the sustainability of m-learning are developed to guide the study in this phase.
Factor analysis
Factor analysis is used to classify the survey items into certain groups. The results of the factor analyses for seven parts are given in Table 6. It shows that our sample adequacy is found to be adequate for factor analyses of all parts with the Kaiser-Meyer-Olkin measure value more than .800. The significance levels of Bartlett’s tests are .000 for all of them; therefore, this test showed that the data are suitable for the analyses. Moreover, the total variance explained is more than .6 in each and this value is seen as satisfactory. Table 7 shows also the number of items and factors.
Results of the Factor Analysis.
Note. KMO = Kaiser-Meyer-Olkin.
According to the rotated component matrixes, 73 sustainability items of m-learning are collected under 20 factors. The frameworks and the hypotheses for the sustainability of m-learning are developed.
Theoretical frameworks and hypotheses
The organizational sustainability items of m-learning are collected under the factors “managerial effectiveness,” “human resource difficulties,” and “consolidations.” Figure 2 shows the organizational items and factors.
Organizational items and factors.
The following hypotheses are developed for organizational sustainability of m-learning:
The financial sustainability items of m-learning are collected under the factors “financial viability” and “effectiveness of financial management.” Figure 3 shows the financial items and factors.
Financial items and factors.
The following hypotheses are developed for financial sustainability of m-learning:
The legal and ethical sustainability items of m-learning are collected under the factors “equity and equality,” “user rights sensitivity,” and “the accuracy of stored information.” Figure 4 shows the legal and ethical items and factors.
Legal and ethical items and variables.
The following hypotheses are developed for legal and ethical sustainability of m-learning:
The pedagogical sustainability items of m-learning are collected under the factors “appropriateness of pedagogical and instructional implementations,” “quality evaluation,” and “pedagogical support for staff, instructors and users.” Figure 5 shows the pedagogical items and factors.
Pedagogical items and variables.
The following hypotheses are developed for pedagogical sustainability of m-learning:
The sustainability items of m-assessment are collected under the factors “sufficiency of assessment media and instruments,” “appropriateness of evaluation criteria,” and “fairness of evaluation.” Figure 6 shows the assessment items and factors.
Assessment items and variables.
The following hypotheses are developed for sustainability of m-assessment:
The psychological sustainability items of m-learning are collected under the factors “Evaluation of user, instructor and staff psychology” and “psychological support.” Figure 7 shows the psychological items and factors.
Psychological items and variables.
The following hypotheses are developed for psychological sustainability of m-learning:
The psychological sustainability items of m-learning are collected under the factors “social presence” and “shared purpose and behaviour.” Figure 8 shows the social items and factors.
Social items and variables.
The following hypotheses are developed for social sustainability of m-learning:
Total of 20 m-learning sustainability factors were identified, and they are shown under the related broad category in Figure 9.
The studied m-learning sustainability factors.
Phase V
In this phase, descriptive analysis, prerequisite tests, and regression analyses are applied to understand the impacts of the sustainability factors of m-learning on the perceived sustainability of m-learning and to test the hypotheses. The final framework is proposed in this phase.
Descriptive analysis
Demographic analyses inform us about the respondents of survey questions. As a result of demographic analyses, the age and m-learning experience statistics of the respondents are given for each sustainability aspect part in Table 8. The number of respondents between the ages of 20 and 30 years is higher due to the number of students. The average percentage of the experiences more than 3 years is 43.1. University information was not compulsory for respondents not to affect their answers. Because the respondents of the social aspect questionnaire were students, there were no respondents higher than 40 years for this aspect.
Demographic Information of the Respondents.
When the descriptive statistics of the sustainability factors of m-learning are examined with respect to the aspects, it can be seen that some issues are at critical levels, and some of them are at the minimum level for sustainability (Table 9).
Issues at the Critical, Minimum, and Maximum Level.
“Finding and retaining staff” as an organizational issue; “IT skills of users,” “cheating,” and “ambiguity about respondent” as assessment issues; and “emotions use,” “personal stories and experience share,” and “humour use” as social issues are at a critical level. Their values are under the value 3. From financial, legal and ethical, pedagogical, and psychological aspects, respectively, “effective risk management,” “informed consent,” “accessibility of support,” and “effectiveness of support” issues have a minimum mean value although they are not at a critical level. Therefore, these issues also should be improved to increase the general sustainability of that aspect. The maximum mean value belongs to “privacy and security of personal information” and “the accuracy of stored information” issues in legal and ethical aspects.
When the dimensional sustainability mean values are examined, it can be seen that all of them are higher than the average value 3 (Table 10). The maximum mean value belongs to “legal and ethical issues” with the value 3.78. The minimum mean value belongs to “financial issues” with the value 3.19. Organizational and financial issues can be perceived as critical issues because their mean values are lower than the perceived m-learning sustainability of related aspects (3.61 for both).
Descriptive Statistics of General Sustainability Responses.
Prerequisite tests
In this study, some prerequisite tests, which are the normality, linearity, multicollinearity, autocorrelation, and heteroscedasticity tests (Poole & O’Farrell, 1971), were applied before the regression analysis. Skewness and kurtosis statistics were checked to prove normal distribution. Residual plots were examined for linearity check. Multicollinearity between independent variables was checked using the tolerance value, and variance inflation factor statistics. Durbin–Watson statistics were used to test autocorrelation. The scatter plot between the standardized residual and the standardized predicted value was examined to detect heteroscedasticity.
Reliability analysis
Cronbach’s reliability test which is the most commonly used technique (Tavakol & Dennick, 2011) is to test internal consistency among items. The results revealed high levels of reliability with Cronbach’s alpha >.9 for all scales according to Kline (2013). Table 11 shows the results of reliability analyses for all survey questionnaires. As a result of reliability analyses, Cronbach’s alpha values are found more than .9 for each survey part.
Reliability Analysis.
Regression analyses
Linear regression analysis is used for testing the hypotheses H4, H7, H12, H17, H21, H24, and H27, while multiple regression analyses are used for other hypotheses.
Table 12 shows that the F values have a significance level of .000. The adjusted R2 values indicate the variance in the dependent variable which is explained by the independent variables. The Durbin–Watson values show that the autocorrelation values and they are between the desired ranges of 1.5 to 2.5.
ANOVA and Model Summary.
Note. ANOVA = analysis of variance.
The results of regression analyses are presented in Table 13. As a result of regression analyses, the significance value of the independent variable “meeting of structural challenges” is more than .1 and it is considered to have not a significant effect on the perceived pedagogical sustainability of m-learning. Therefore, H14 is not substantiated by this test. The significance values of all independent variables are less than .05 except one. The variable “appropriateness of pedagogical and instructional implementations” is significant at the .1 level with the value of .088. The other independent variables have p values which are less than .05. Their impacts on the related dependent variables are considered significant. Therefore, only H14 is not accepted.
Regression Analysis Results.
Significant at the .1 level.
As a result of the analyses, the final model for the sustainability of m-learning is presented in Figure 10. The model consists of 19 factors with seven dimensions.
The proposed model for sustainability of m-learning.
Discussion
Adoption or success is not sufficient for a sustainable m-learning project. M-learning initiatives should also meet the educational intents, deal with difficulties, make progress or maintain a certain condition, and be prepared for changes and risks. The literature is reviewed, and experts are consulted for the sustainability of m-learning considering these necessities. Hypotheses are developed with the obtained factors and tested with the survey data. All hypotheses are accepted except one which is “meeting of structural challenges has an impact on the perceived pedagogical sustainability of m-learning.” Determination of the environment and activities and development of the platform require considering different learning styles (Villaverde et al., 2006), and metacognition (Terras & Ramsay, 2012). The readiness level of learners was also suggested by experts as an item which can affect the factor “meeting of structural challenges.” The items “consideration of different learning styles,” “essentiality of metacognition,” and “consideration of readiness level” are classified under the title “meeting of structural challenges.” They are also classified under the same component in the rotated component matrix, and there is no increase in the overall alpha value when one of them was deleted; therefore, they were not eliminated. However, there may be additional items that can be grouped under this factor. To reveal these items, the number of experts can be increased for future studies, literature findings can be reviewed by experts, or the geographical area of the research can be expanded. Thus, some limitations of this study are resolved.
If the results of the factors that have a significant impact on sustainability will be discussed, because the results overlap with the literature, discussing their coefficients may provide some insights that may be useful for other m-learning practitioners. When the result of the organizational factors is examined, it can be seen that consolidation has the highest coefficient value. The combination of several business units or different companies into a single, larger organization is useful to make better use of resources and more effective organizational structure, reduce redundant processes, and improve service delivery. It provides budget-saving and also affects financial statements. Because financial sustainability is a prerequisite for a corporation’s survival, it is an important one for an initiative (Steurer et al., 2005). The coefficients of the financial sustainability factors may result from this reason. The other important factors are related to user rights and assessment media and instruments. The lesson here is that effective user rights sensitivity can be a decisive factor for legal and ethical m-learning sustainability, whereas the study indicates that the sufficiency of assessment media and instruments is perhaps the most important part of m-assessment. According to the results, pedagogical and psychological supports for users are two of the most influencing factors for sustainability, and the literature is in agreement with the results (James, 2008; Stansfield et al., 2009). Finally, the success of the project owes much to the social presence, which is emphasized by many researchers such as Lowenthal (2010b), Aragon (2003), Tu et al. (2016), Alsadoon (2018), and Ooi et al. (2018).
The factors in the final proposed model affect the sustainability of m-learning; however, some of them are required more attention according to descriptive statistics because the average value of the perceived sustainability of m-learning in sample universities is reduced by them.
One of them is “finding and retaining staff.” Berge and Kearsley (2003) stated that it is one of the important organizational challenges for e-learning initiatives. According to heads in UCDE, system staff, design and development staff, and instructors are not willing to be involved in a system if it is not a well-designed and working system. Because educational content is critical for the widespread uptake of m-learning, m-learning requires the preparation of appropriate content and material for mobile devices and the involvement of the instructors. However, the instructors’ perspectives of m-learning are negative. They do not know to design and develop m-learning content and materials, and there is no standard for content and materials. Therefore, this pedagogical reason causes the organizational issue about finding and retaining staff, and m-learning course design process that does not include the support of instructors leads to failure.
Moreover, the assessment issues “IT skills of users,” “cheating,” and “ambiguity about respondent” are some of the critical issues. Some students try to engage in m-learning, while they are considered as digital natives. The m-assessment environment requires a certain level of IT skills. Therefore, some students are struggling when they are assessed, and inequity may occur in assessments. Dyson et al. (2017) emphasized the issue of cheating and the experts stated that the ambiguity about the respondent during an m-assessment process is also a problem for a fair assessment. According to regression analyses, fairness of evaluation has significant impact on assessment sustainability of m-learning as the experts point out. Therefore, the lack of fairness may cause m-learning not to be preferred.
Social interaction is another concern for m-learning initiatives. According to analysis, the levels of emotion use, personal stories and experience share, and humor use are low in sample universities. Aragon (2003) stated that these issues may lead students, instructors, and staff to antisocial behavior.
In addition to meeting educational intents, financial sustainability is also important for a sustainable m-learning initiative. Expensive m-learning projects are more likely to stall and fail than less expensive projects (UNESCO, 2011). Many m-learning initiatives start with huge financial investment, and effective financial risk management is also required.
Current literature emphasizes the support for students and instructors (Sánchez et al., 2013; Stansfield et al., 2009). According to the results of this study, the accessibility and effectiveness of support and feedback for stakeholders, and communications among them are also essential, especially for the pedagogical and psychological sustainability of m-learning.
The issue “informed consent” is a problem in m-learning for sample universities. According to Lally et al. (2012), explaining the nature and extent of a virtual environment might be difficult. However, Traxler and Bridges (2005) stated that this may lead to participants not being able to obtain consent or to be informed of the activities.
As a result, sustainable m-learning has many dimensions, including organizational, financial, legal and ethical, pedagogical, assessment, psychological, and social dimensions. Not only designing and developing a system but also designing an ecosystem with considering these dimensions of sustainability is required for sustainable m-learning.
Conclusion
This study intends to specify the sustainability factors of m-learning and the research question is “What are the factors that can impact the sustainability of m-learning?.” Interviews are conducted with heads in UCDE to obtain additional sustainability factors to the literature-based factors. Survey research is conducted with administrative, system, design, and development staff in UCDE, and instructors and students in distance education universities to test the impacts of the obtained factors on the sustainability of m-learning. As a result of the data collection and analyses, a model for sustainability of m-learning is proposed. Moreover, the current sustainability statuses of m-learning at the universities are discussed.
This study can be a roadmap for m-learning initiatives (boards of education, educators, policymakers, and staff) in sustaining effective m-learning in this period when mobile technologies have become the world of people and the importance of m-learning increased with the COVID-19 pandemic. Universities and other m-learning initiatives must meet the increasing demand with sustainable education policy. This study can bridge the gap in the sustainability issue of m-learning literature with many perspectives and provide a guideline to assist m-learning initiatives for sustainability.
This study assumed that the respondents accurately responded to the survey about the current m-learning status of their initiative. The geographic area of the investigation may be a limitation for the generalizability of the study. Response bias may be existed due to an internet-based survey without incentives, and the number of experts can be increased for future studies. Furthermore, the databases browsed could be compared to understand how the same literature would be covered (overlap) by different databases and to measure the amount of information that would be lost if a particular database was destroyed (Pulgarín-Guerrero & Escalona-Fernández, 2007). In future studies, the sustainability aspects of m-learning can be studied in different countries and for different business institutions. Moreover, other aspects of sustainability of m-learning can be investigated for m-learning.
Footnotes
Appendix A
Appendix B
Summary of Interviewee Profiles.
| Expert ID | Occupation | Sex | Age (years) | Years of distance learning experience | Education degree |
|---|---|---|---|---|---|
| 1 | Deputy Manager in UCDE | Female | 58 | 14 | Ph.D. |
| 2 | Professor in UCDE | Female | 40 | 10 | Ph.D. |
| 3 | Distance Learning Coordinator | Female | 38 | 12 | B.S. |
| 4 | Program Coordinator of Online Training Unit | Female | 41 | 6 | B.S. |
| 5 | Distance Education Instructional Designer | Female | 34 | 9 | B.S. |
| 6 | Professor in UCDE | Male | 43 | 13 | Ph.D. |
| 7 | Educational Technology Deputy Manager | Female | 31 | 9 | M.S. |
| 8 | Professor in UCDE | Male | 44 | 10 | Ph.D. |
| 9 | Distance Learning Coordinator | Male | 65 | 10 | Ph.D. |
| 10 | Manager in University Distance Education Center | Male | 41 | 12 | Ph.D. |
| 11 | Distance Education Project Manager | Male | 40 | 11 | M.A. |
Note. UCDE = university centers for distance education.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval
Institutional Review Board for Research with Human Subjects (
