Abstract
Technology brings tremendous changes in education because it is a system that automates all educational institutions and academic performance. Therefore, the study examines the effect of technology’s system, information and service quality on faculty, operational and university performance. To explore the more contextual factors, this study empirically and theoretically tested a proposed model by the D&M theory of IS among UAE universities. The study employed mixed-method research using a sequential explanatory research design. Using a designed survey questionnaire, the study targeted 512 faculty members and conducted 10 semi-structured face-to-face interviews with faculty members of 27 UAE universities. The results reported that system, information, and service quality significantly influenced faculty performance. In addition, system quality strongly affects faculty performance, which is the most necessary part of successful technology implementation. Faculty performance significantly influences operational and university performance; surprisingly, it has the strongest influence on operational performance. In turn, operating performance has a significant impact on university performance. The study further identified four contextual factors, that is, external, individual, organizational, and technical. The study put the novel ideas by contributing performance-level measures that support Delone and Mclean’s IS success model to successfully operationalize the university’s actual performance. The research uniquely extends the D&M IS success model to assess technology implementation success at individual, operational, and organizational levels within UAE universities, touching previously unexplored areas of post-implementation evaluation. University management in the UAE should prioritize enhancing service, system, and information quality to bolster faculty performance, leading to improved operational and overall organizational outcomes.
Plain Language Summary
Technology brings tremendous changes in education because it is a system that automates all educational institutions and operations; therefore, the study examines the effect of technology’s system, information and service quality on faculty, operational and university performance. To explore the more contextual factors, this study empirically and theoretically tested a proposed model by the D&M theory of IS in UAE universities.
Keywords
Introduction
Educational technologies (i.e., in e-learning, and m-learning) have significantly reshaped faculty performance, underscoring their centrality in modern education (Almaiah et al., 2018; Almaiah & Alyoussef, 2019). While studies have delved into the effects of course design and teacher characteristics on e-learning utilization, there is a marked interest in e-learning, especially when contrasting intentions between early adopters and latecomers (Almaiah et al., 2018, 2022). The COVID-19 pandemic showed some critical challenges in e-learning system usage, and studies from regions like Jordan spotlighted issues within m-learning. Integrating these platforms also sees a blend of multifaceted factors at different stages of m-learning application development and challenges in e-learning system implementation (Almaiah & Almulhem, 2018). Technological advancements, including AI, virtual reality, and IoT, rapidly reshape education. With its advanced education sector, the UAE exemplifies this shift (Andrew et al., 2018). Despite economic challenges, universities increase IT budgets (Lutfi et al., 2022), striving for efficiency amidst rising competition (Bag et al., 2021; Wang et al., 2014). These technologies range from hedonic systems for enjoyment to utilitarian ones that boost performance (Van Der Heijden & Wouters, 2004). They centralize university operations, streamlining the management of educational services.
Information Systems (IS) in education, characterized by pre-defined user rights, facilitate educational functions within universities through a centralized network system (Andrew et al., 2018). End-users access and share data across departments, enhancing operational performance. Adopting such technology promotes flexible learning, breaking spatial and temporal barriers (Al-Awidi & Alghazo, 2012; Alhashmi et al., 2019). In the UAE’s economy, universities leverage technological advancements to capture a significant economic share, aligning with their strategic objectives. Global institutions, in contrast, seek systems that offer scalability and multi-lingual, multi-channel support (Al-Awidi & Alghazo, 2012). Notably, key UAE educational institutions, like Khalifa University, the American University of Sharjah, the University of Sharjah, and the United Arab Emirates University, have transitioned to excellent online platforms since the decade’s onset (Alhashmi et al., 2019), setting a benchmark for other institutions to follow. Figure 1 shows the different stages of implementing technology life cycle in UAE universities:
Different stages of implementing technology life cycle in university.
Initial issues such as organization beliefs and attitude toward technology transformation, checking available sources, applications and software package selection, estimated cost and paybacks, etc., are discussed in the first stage. In the second stage, issues such as data cleansing, staff training, infrastructure, and critical success factors that affect the university’s success (Al Kurdi et al., 2020; Salloum, 2018) are discussed. The last stage is the post-implementation stage that evaluates the success/effectiveness of technology implementation on the individual, operational and organizational performance of higher educational institutions (HEIs). Technology implementation success, post-implementation success and effectiveness are interchangeably used in information system literature (Sewandono et al., 2023). A study by Ifinedo (2006) claimed that improved performance at various organizational levels proves system success. The theory of IS by DeLone and Mclean (2003) was used by many researchers to evaluate the effectiveness of any information system. Similarly, the current study has also applied the captioned theory to assess the success of technology implementation in UAE universities.
Several factors influence the success of technology implementation in UAE universities, yielding varying outcomes across different Higher Education Institutions (HEIs). The identified factors include (1) external factors, (2) Individual (faculty) factors, (3) Organizational (university) factors, and (4) Technical factors. While numerous studies have globally examined the impact of technology on staff satisfaction and organizational performance, such as by AlHamad (2020), the research has primarily centered on the benefits of technology at the individual and organizational tiers. No research has probed into the success of technology implementation across the individual, operational, and organizational strata, particularly in the UAE context. E-learning, as a subset of technology transformation, has been explored in several studies (Riandi et al., 2021; Sewandono et al., 2023). For instance, Al-Azawei et al. (2016) found that e-learning positively impacted the technology usage in a relatively short time. Another study by Abulibdeh and Syed Hassan (2011) indicated that technological advancements led to heightened university adaptability and profitability. Despite these insights, there remains a significant research gap. The post-implementation phase of technology, especially using the IS success model, still needs to be explored. Moreover, while there is a focus on the growth of e-learning and its implications for university performance, comprehensive research on the post-implementation success of broader technology transformations in education needs to be discussed more. Most critically, existing studies, particularly in the UAE, have overlooked the contextual factors, such as organizational (universities), individual, technical, and external factors, that determine the success of technology implementation. Therefore, a deeper exploration into technology’s success across various levels, considering these influencing factors within UAE’s HEIs, remains an urgent research needs.
Very few studies have evaluated e-learning facilities and academic performance in the UAE universities (Abulibdeh & Syed Hassan, 2011; Al-Azawei et al., 2016; Al-Emran et al., 2016; AlHamad, 2020). Further, studies have only examined the success of information systems at the individual level (Ahmad & Daghfous, 2010). Hence, the success of information systems at the operational level has never been studied. The measurement of information system success at the operational level is fundamental because this is the stage where integration and automation of all operations are executed. Al-Mudimigh et al. (2009) pointed out the importance of information systems at the operational level, which have been addressed in previous studies. However, this study evaluates the post-implementation success of the technology implementation in five universities (i.e., HEIs) in the UAE. The present study is based on DeLone and Mclean’s (2003) theory of IS because the study used this model to evaluate the success of technology implementation in UAE universities. In UAE, very limited studies have examined the success of technology implementation in UAE HEIs, particularly in universities. Further studies focused on pre-implementation issues of information systems in corporate sectors (Ijaz et al., 2014; Shad et al., 2012). Unfortunately, the measurement of information system (IS) success in the universities of UAE has never been studied. This research paper likely contributes to the existing literature by evaluating the success of technology in the post-implementation phase of the university information system. Therefore, the main objectives of the current study are:
to examine the impact of technology-based system quality, information quality, and service quality on faculty performance in UAE universities,
to examine the impact of faculty performance on operational performance and university performance in the UAE universities,
to examine the impact of operational performance on university performance in UAE universities, and
To explore the external factors affecting the successful implementation of technology in UAE universities.
The study followed the sequential explanatory research design. In the first stage, the target population distributed the survey questionnaire to collect the quantitative data. After collecting quantitative data, the semi-structured interviews were conducted to compare and validate the results with quantitative data. The target population was faculty members from 27 universities in UAE. The study followed the convenient sampling strategy because this sampling strategy supports the researcher in getting the responses from the readily available respondents. To examine the SEM analysis, the current study used PLS-SEM software to test and validate the quantitative portion of the study; however, to try the qualitative method, NVIVO version 11 was used.
Literature Review and Theoretical Model
This literature review supports the development of theoretical framework by creating hypotheses (Figure 2). There are different studies that have been done in the universities such as Camilleri (2021) explored the evaluation methodologies and metrics to measure higher education institutions’ service quality and performance. In light of the challenges posed by the COVID-19 pandemic, there is an implied need to re-evaluate the parameters that define quality, incorporating the realities and demands of remote learning and educational experiences. Riandi et al. (2021) proposed a conceptual model that underscores user satisfaction as a pivotal mediator between e-learning services/system quality and students’ performance. The study suggests that the perceived quality of e-learning services directly influences student satisfaction. Jameel et al. (2021) found the pivots around “e-satisfaction,” which is based on the quality of e-services among university students. The study findings indicate a direct correlation between the perceived quality of e-services and the satisfaction students derive from these services. In addition, Ramírez-Hurtado et al. (2021) delved into the realm of online teaching service quality during the challenging phase of the COVID-19 pandemic. There is a need to devise important measures to assess the quality of online teaching services, as it plays a significant role in student satisfaction and learning outcomes, particularly in the constrained environment brought about by the pandemic.
Research model of technology success & performance.
In a recent study by Sewandono et al. (2023), they explore the performance expectancy of E-learning in higher education institutions. Their focus on the Indonesian context provides a distinctive perspective, given unique educational landscape and its infrastructure and resource challenges. In addition, Abdullah et al. (2021) embarked on an insightful exploration of this concept. They underscore the pivotal role of internal service quality in shaping nurses’ job outcomes, specifically emphasizing the mediating role of employee well-being. Finally, the study develops the theoretical model based on the literature pieces of evidence:
Application of D&M Model for Technology Implementation
The model by DeLone and Mclean (2003), offers a holistic approach to evaluating e-learning system success, as Almaiah and Almulhem’s (2018) study shows. Unlike the TAM, which Almaiah et al. (2016) extended to address quality features in mobile learning, and UTAUT, used by Almaiah et al. (2019) to understand acceptance, the DeLone and McLean model delves deeper. It addresses the initial acceptance of a system and encompasses post-adoption factors like user satisfaction and perceived benefits, providing a user perspective on user experience. Additionally, Huang et al.’s (2017) bibliometric analysis underscores the model’s widespread application, further validating its effectiveness over TAM and UTAUT for a more in-depth e-learning system analysis. Lutfi et al. (2022) found that system quality and information quality significantly influenced IS usage, while service quality did not. Furthermore, IS use and user satisfaction positively influenced the sustainability of decision-making in Jordanian organizations.
Mtebe and Raisamo (2014) used the D&M model to evaluate the learning management systems (LMS) used in higher education in terms of the quality of the system, the quality of the information, and the level of user satisfaction. Mtebe and Raisamo (2014) discovered a positive correlation between the quality of the system and the quality of the information and user satisfaction. As well, TAM model was used to evaluate the successful implementation of technology in higher education (Almaiah et al., 2016). In addition, Huang et al. (2017) conducted a bibliometric analysis of studies that applied the D&M model in various settings, including the higher education sector. Lutfi et al. (2022) discovered that the D&M model had been utilized frequently in research conducted within higher education to evaluate the efficiency of information systems. Alkhalaf et al. (2021) to analyze the factors that influence the application of technology in higher education utilized the D&M model. Alkhalaf et al. (2021) examined that the quality of the system, the quality of the information, the quality of the service, and the individual performance of the users were significant antecedents of the use of technology in higher education. Nguyen (2018) to research the factors influencing the acceptance and utilization of e-learning systems in developing nations utilized the D&M model. The findings showed that IS was successful on individual and organizational levels, but they ignored the operational level, which is where the impacts of individual performance are transformed into the performance of the organization.
Importance of Operational Performance
The operational performance refers to the operations such as institutional structure, backup recovery, performance tuning, maintenance, security control, and systems control (Mtebe & Raisamo, 2014). These operations help firms to reap full glories with the least resources. A six items scale developed by Ravichandran et al. (2005) has been used to define this variable. In addition, Hyvönen (2007) argued that operational performance is positively related to organizational performance. Duarte (2011) pointed out that operational performance through its different techniques, like TQM, had a significant positive influence on organizational performance. The D&M model in previous studies was tested and cited in IS research studies, but almost all the research studies overlooked the importance of IS success at the operational level. Petter et al. (2008) supported this concept. After analyzing the literature on IS success theory of D&M, studies had only evaluated the success of IS on individual and organizational levels. Different economic benefits are associated with Information systems. It speeds up the operating performance in organizations, and its user’s o display positive performance compared to non-users. One study by Tung et al. (2008) also established that the Information System improves organizations’ business volumes and operational efficiency.
Theoretical Framework/Model
System Quality and Individual (Faculty) Performance
Various studies have tested and established a positive linkage between system quality and individual performance (Diar et al., 2018; Peters & Panayi, 2016). Sedera and Gable (2004) employed confirmatory factor analysis, and the results showed a strong link between service quality and individual performance. Similarly, Wickramasinghe and Karunasekara (2012) used a survey questionnaire approach to assess the post-implementation impacts of IS at different managerial levels in Sri Lanka. The findings supported the hypothesis that IS performance offers problem-solving support at various individual levels. Several empirical studies (Hsieh & Wang, 2007; Mohammadyari & Singh, 2015), which were analyzed and linked in a positive manner by Peters and Panayi (2016) in an excellent meta-analysis on the assessment of IS publications, looked at the link between service quality of the system and individual net benefits. In addition, Ifinedo et al. (2010) carried out a cross-sectional field study of 109 businesses located in two countries in Europe. Ifinedo et al. (2010) claimed that there is a positive relationship between the quality of the system and the effectiveness of the working in businesses. It was also contended that a positive association exists between individual performance and the quality of the IS system (Al-Fraihat et al., 2020). Based on the following literature review, the study proposes the research hypothesis:
Information Quality and Individual (Faculty) Performance
Evidence from a number of empirical studies points to a positive link between the quality of the information and an individual’s level of performance (Al-Fraihat et al., 2020; Mohammadyari & Singh, 2015). According to the findings, there is a significant link between the quality of the information and individual performance. In a similar vein, Bharati (2006) conducted research and concluded that the quality of information has a significant influence on quality of work, which, in turn, led to timesavings and better decision-making. In addition, Wu and Wang (2006) found that there was a significant link between the quality of the information and the net benefits that users received. Kositanurit et al. (2006) conducted research to control whether or not there is a relationship between the quality of the information and the individual performance of users. According to Kositanurit et al. (2006), there is a significant positive link between the quality of the information and an individual’s performance. According to the findings of a groundbreaking investigation of digital libraries, the quality of the information that can be retrieved and the perceived usefulness of that information are positively related to one another (Mohammadyari & Singh, 2015). According to Wixom and Todd’s (2005) findings, the quality of the information was found to have a positive correlation with a purported reduction in the amount of time and effort required to make decisions regarding research. According to the results of a study conducted by Hamakhan (2020) in the public sectors of Norway and Denmark, he found that the quality of information provided by IS had a significant positive relationship with individual performance. Based on the results of these studies, the current study proposes the research hypothesis:
Service Quality and Individual (Faculty) Performance
Service quality in the education context means service and support by internal IT departments of the universities. This construct has five dimensions of the revised SERVQUAL model by Parasuranman (Zeithaml et al., 1988), including reliability, assurance, responsiveness, tangibility and empathy. IS service quality means the procedures of skills, knowledge, and capabilities of the support staff, that is, proper follow-up, competencies, responsiveness and confidentiality (Cidral et al., 2018). Using different methods, scholars established a strong link between system quality and individual performance (Almutairi, 2005; McGill & Klobas, 2005; Wixom & Todd, 2005). Tsai et al. (2011) established a significant positive link between the service qualities of consultants and individual performance by using the integrated SERVQUAL model. Similarly, Masrek et al. (2007) claimed that service quality and individual performance had a significant positive association. A cross-sectional quantitative survey by Ifinedo et al. (2010) of 109 firms in two countries of Europe established the progressive effect of IS service quality on individual performance. Following the literature pieces of evidence, the study proposes that:
Individual (Faculty) Performance and Operational Performance
The ability of an individual performance to make decision-making that are effective and timely, increase productivity, learn and grow, and show improvement in their day-to-day activities are the criteria that are used to evaluate their performance. Performance practices in an organizational setting is typically alienated into three levels: strategic performance leadership, operational performance management, and individual performance strategic planning (Brudan, 2010). A number of studies (Almutairi, 2005; Cidral et al., 2018) have established a significant and positive link between an individual’s performance and the performance of their operations. In a study that was conducted by Brudan (2010), it was asserted that individual performance and operational performance had been significantly associated with one another. Based on the literature pieces of evidence, the study hypothesizes that:
Operational Performance and Organizational (University) Performance
Operational performance is the measurement against the standard or prescribed effectiveness and efficiency indicators, whereas organizational performance comprises actual results. A firm’s operational performance is the part of the strategic planning, educational instrument, security control, performance indicators, maintenance, and systems control (Ravichandran et al., 2005). These operations help firms to reap full glories with the least resources. Richards (2009) claimed that organizational performance comprises three main factors, that is, individual performance, product performance and shareholder intentions to participant. Several studies have observed the association between operational performance and individual performance. Hyvönen (2007) argued that operational performance is positively associated with organizational performance. Duarte (2011) pointed out that operational performance through its different techniques, like TQM, significantly and positively influence organizational performance. Therefore, the study hypothesizes that:
Individual (Faculty) Performance and Organizational (University) Performance
Organizational performance is the firm’s performance in the context of education quality and service quality, operational proficiency, sales revenue, user satisfaction, competitiveness, and customer relations (Chang et al., 2005). Various studies have examined the positive link between individual performance and organizational performance. For example, a study in Taiwan by Lin et al. (2006) concluded that individual performance is linked with organizational performance. Similarly, Lee and Lee (2012) assessed the success of IS in different organizations. In the finding, they argued that individual and organizational performances are positively associated (Mehralian et al., 2017). Furthermore, Ifinedo et al. (2010) claimed a significant and positive link between individual and organization performance after conducting a cross-sectional nature survey in European countries to measure IS system success. Similarly, Lin et al. (2006), while measuring the success of IS systems based on the D&M model, found a significant and positive association between the individual and financial performance of organizations. The study also proposes that:
Research Methodology
Research Approach
The study used mixed-method research following a sequential explanatory (i.e., Quantitative-to-Qualitative) research design (Cresswell, 2012). In the current study, the cross-sectional (i.e., convenient sampling approach) survey is selected to examine the technology success and implementation at faculty, operational and university level in an entire period from five UAE universities. The study is divided into two parts or stages. Pre-testing of the survey questionnaire was carried out in the very first stage to check the distribution and editorial structure of the questionnaire, to guarantee accurate responses, and to avoid ambiguous, unclear, or confusing questions (Babbie, 2007; Cresswell, 2012). Subsequently, the administration of the survey questionnaire was carried out in the second stage as part of the primary study. The stages of research design are evaluated in Figure 3.
Summary of research design.
Pre-Testing of Questionnaire
According to Cresswell (2012), a pilot test of the survey questionnaire is required in order to establish face-to-face and content validity, as well as to enhance the questions, the format, and the scales. Validity refers to the accuracy with which a construct is measured, as well as the degree to which a scale is able to measure the construct that it is intended to measure (Pallant, 2011). The term “content validity” refers to the manner in which the indicators measure various facets of the concepts (Adams et al., 2007). Pretesting was done to verify the relevancy of each question associated with the study hypotheses variables. This helped to ensure that each item was suitable, which contributed to the justification of the face and content validity of the measurement. In the current study, pretesting was conducted by sharing the questionnaire with expert technology implementation teams in UAE universities. The changes were incorporated into the final questionnaire.
Sample and Sampling Technique
The current study has employed a convenience sampling technique to collect the data (Sekaran, 2009). The study’s target population is faculty members from 27 universities in UAE. Sekaran (2009) considered a sample size larger than 30 and less than 500 appropriate for testing the research hypotheses.
Data Collection and Procedures
The current study has adopted the survey monkey to calculate a sample size of 500 respondents. Since the present study is cross-sectional, the data were collected in a natural environment by visiting the university departments from UAE universities. The current study applied a convenient sampling technique to collect the data. A total of 512 questionnaires were distributed to the respondents, out of which 373 questionnaire responses were received correctly. The remaining 139 questionnaires were not completely filled, so it was better to remove them, so the study did not add incomplete surveys. 373 responses were received, yielding a response rate of 70%. Later, the study conducted 10 semi-structured interviews with the faculty members, and the interview time was almost 20 min. Creswell (2014) suggests that in qualitative research, minimum six interviews can provide rich and comprehensive data; however, the study conducted 10 interviews to support the quantitative findings. One semi-structured interview has been done with one faculty member of each of 10 UAE universities. The researcher got permission and followed ethical codes from the participants before conducting qualitative interviews. The researcher used English for the semi-structured interviews because the respondents were well-qualified professionals. The researcher used a tape recorder to record the voice because the respondents did not allow the researcher to record videos. Finally, the researcher transcribed English voice interviews into MS word and started coding in Nvivo 11. The current study considers the four steps Braun and Clarke (2006) recommended to ensure the validity of qualitative data.
Measurement Scales
The study adapted the measurement scales from the previous studies. Each scale has the good validity and reliability; therefore, the study adapted the scales checking by two professors and two technical experts in the field. The survey questionnaire was developed by the recommendations of four professionals because the previous items were adapted accordingly. A seven items scale has been used for system quality adapted from Gable et al. (2008) and Sedera et al. (2004). In addition, service quality means the technical support and services to end-users from the internal MIS department. A five items Likert scale of SERVQUAL model in the study of Parasuraman et al. (1988) which is further validated in the UAE context. The model provides the basis for measuring service quality features, that is, reliability, responsiveness, tangibles, assurance and empathy in the services of the technology implementation team. Information quality refers to the quality generated by technology systems. It consists of features, that is, understandable, meaningful, brief, relevant, available, and usable features. A seven items scale has been used for system qualitys developed by Gable et al. (2008) and Sedera et al. (2004). Individual performance refers to the performance after using the technology implementation system. It consists of various characteristics, that is, individual creativity, productivity, learning, quality of decision making, benefits in individual’s tasks and time-saving in duties and functions. Previous studies developed a six items scale (Gable et al., 2008; Sedera et al., 2004) that has been adapted to measure individual performance. Operational performance refers to the operations helping firms to reap full glories with the least resources. A six items scale developed by Ravichandran et al. (2005) has been used to measure operational performance. Organizational performance refers to different performance parameters of financial institutions, that is, profitability, productivity, cost efficiency, competitiveness and utilization of organizational resources. A seven items scale developed by Gable et al. (2008) has been adapted to measure organizational performance. All measurement scales have good reliability, including Cronbach alpha and composite reliability > .70 and validity, including AVE > .50 and Factor-loadings > .70. Finally, all the measurement items are measured on a 5-point Likert scale ranging from 1 = strongly disagree-5 = strongly agree.
Data Analysis
To test the hypotheses and examine the proposed model, this study applied a widely known technique of partial least squares (PLS) based on component structural equation modeling (SEM) using Smart PLS 3.3.3 (Richter et al., 2020; Ringle et al., 2020). The study used covariance-based SEM to assess the reliability and validity of the measurement constructs (Henseler et al., 2015). The PLS-SEM technique provides accurate and statistically robust results even with a small sample size and complex model (Sharif et al., 2022). Statistical tests were conducted to draw the findings of survey data and the thematic analysis was conducted to draw the findings from interviews. For qualitative data analysis, this study used Nvivo version 12, and the study follows the six steps approach of Braun and Clarke (2006). The fact that this software offers a variety of search tools, in addition to linking, marking up, and reorganizing the data, was a significant factor in our decision to use it.
Quantitative Findings
Demographic Information
Table 1 presents demographic information of the respondents. Out of the 373 respondents, 44% (164) were from public universities, and 56% (209) were from private universities. Among the respondents, 55% (205) reported universities using Blackboard, and 45% (168) reported using Ellucian. Among the respondents, 25.7% (96) were in lower-level management, 64.9% (242) were in middle-level management, and 9.4% (35) were in top-level management. Among the respondents, 88.5% (330) were male, and 11.5% (43) were female. The fifth and sixth categories are individual experience and company experience, respectively. For each category, respondents were asked to select one of four options: less than 1 year, 1 to 3 years, 4 to 6 years, or more than 6 years. Among the respondents, 9.1% (34) reported less than 1 year of individual experience, while 23.3% (87) reported more than 6 years. For company experience, 4.6% (17) reported less than 1 year, while 42.9% (160) reported more than 6 years. Finally, 1.6% (6) reported graduation as their highest qualification, while 66.2% (247) reported a master of philosophy degree.
Demographic Information.
Assessment of Measurement Model
Convergent Validity and Reliability
The present study applied the PLS algorithm technique with 5,000 sub-samples to scrutinize the research model to measure construct reliability and validity, including convergent and discriminant validity (Hair et al., 2020; Krishnan et al., 2011; Richter et al., 2020). The study applied a series of algorithm run to meet the threshold values. First, factor loadings and average variance extracted (AVE) extent convergent validity, whereas cross-loadings and Heterotrait-Monotrait (HTMT) ratio measure the discriminant validity of the constructs (Hair et al., 2020; Krishnan et al., 2011). Cronbach alpha and composite reliability also extent the construct reliability (Richter et al., 2020). The researchers recommended that the value for factor loadings > 0.70 and reliability > 0.70 should be higher than 0.70 (Hair et al., 2020; Krishnan et al., 2011; Sharif et al., 2022). The study found that factor-loadings > 0.70 and AVE > 0.50 value of each latent construct were higher than the threshold values. All constructs have AVE values above the acceptable threshold of 0.5, with values ranging from 0.582 to 0.712. The highest AVE values were observed for service quality and operational performance, while the lowest was observed for system quality.
The composite reliability values are all above the acceptable threshold of 0.7, with values ranging from 0.848 to 0.925, indicating good internal consistency. Similarly, Cronbach’s alpha values are all above the acceptable threshold of .7, with values ranging from .573 to .925. Table 2 shows that the study meets the threshold values for construct validity and reliability; therefore, there was good reliability and validity. Overall, the results of the factor analysis suggest that the constructs are reliable and valid measures of their respective constructs. Finally, the study proved construct validity and reliability.
Construct Validity and Reliability.
Notes. F.L = factor loadings; CR = composite reliability; AVE = average variance extracted; C.A = Cronbach alpha.
Discriminant Validity
First, cross-loadings are used to evaluate the degree to which a specific indicator is related to more than one latent variable in a measurement model (Sharif et al., 2022). This can be problematic for the discriminant validity of the model (Table 3). It is recommended that the magnitude of the cross-loadings be less than the squared correlation that exists between the corresponding latent variables in the model (Krishnan et al., 2011). This is the case because smaller cross-loadings are associated with better predictive accuracy. Overall results do not indicate severe issues with discriminant validity
Cross-Loadings.
Second, the HTMT (Heterotrait-Monotrait Ratio) is used to assess the discriminant validity of a measurement model (Table 4). Discriminant validity refers to the degree to which a construct is distinct from other constructs in the model (Henseler et al., 2015). Generally, HTMT value below the threshold of 0.85 indicates good discriminant validity, while a value above 0.85 suggests poor discriminant validity (Henseler et al., 2015). In this study, all the HTMT values are below the threshold of 0.85, indicating good discriminant validity among the constructs. Specifically, the highest HTMT value is 0.833 between organizational (university) performance and information quality, which is still below the threshold (Hamid et al., 2017). Table 5 demonstrates that the same discriminant validity is satisfied throughout the research.
HTMT Ratio.
Structural Equation Modeling (SEM).
Path Model Assessment
Figure 4 and Table 5 show the results of the SEM model comprising standardized regression analysis, t-value and p-value and R-square statistics for endogenous constructs (Hair et al., 2020; Peng & Lai, 2012). Table 5 presents the beta coefficients, t-values, and p-values for the six hypotheses tested. Hypothesis 1 (H1) proposed that System Quality has a significant and positive influence on Individual (faculty) performance, and the results show a significant positive influence (beta = 0.279, t-value = 5.043, p < .001), indicating that a higher level of system quality results with better individual faculty performance. Hypothesis 2 (H2) proposed that Information Quality has a significant and positive influence on Individual (faculty) performance, and the results show a significant and positive influence (beta = 0.227, t-value = 3.323, p = .001), indicating that a higher level of information quality is associated with better individual faculty performance. Hypothesis 3 (H3) proposed that Service Quality has a significant and positive influence on Individual (faculty) performance, and the results show a significant positive influence (beta = 0.284, t-value = 5.114, p < .001), indicating that a higher level of service quality results with better individual faculty performance. Hypothesis 4 (H4) proposed that Individual (faculty) Performance has a significant and positive influence on Operational performance, and the results show a significant and positive influence (beta = 0.641, t-value = 17.489, p < .001), indicating that better individual faculty performance results with better operational performance. The SEM analysis shows a positive beta coefficient of 0.331 and a t-value of 5.907, which is statistically significant at p < .001. This indicates a strong positive and significant influence of individual (faculty) performance on organizational (university) performance.
Structural equation modeling.
Overall, the study suggests that improving service quality, system quality, and information quality can lead to better individual (faculty) performance, which in turn can improve operational and organizational (university) performance. These findings can be helpful for universities and other educational institutions in improving their performance by focusing on improving the quality of their services, systems, and information, as well as the performance of their faculty members.
A Sequential Process From Quantitative to Qualitative
The empirical findings of the present study have proved all the hypotheses of the current study. These findings established a significant and positive influence of the quality dimension of technology implementation and success on the various performance perspectives, that is, individual performance, operational performance and organizational performance. From the findings, this study examined the post-implementation success of technology implementation in UAE universities. To answer the second research question, which is about the contextual factors that affect the success level of technology implementation and success in UAE universities? This study conducted an in-depth interview to report the findings in the next section.
Critical Success Factors: The Qualitative Perspective
The early section of this study already provided an extensive discussion on these factors, which are divided into four categories, that is, external factors, individual factors, organizational factors and technical factors.
The study identified four contextual factors by conducting face-to-face semi-structured interviews. The study coded structured interviews into nodes. Figure 5 shows propositions developed based on the strong theoretical background, which indicates the expected relationship between the contextual factors and the success level of the technology implementation in the following manner:
External factors for implementation success
Individual factors for implementation success
Organizational factor for implementation success
Technical factors for implementation success
Critical success factors of technology implementation.
External Factors and Technology Implementation Success
Economic Conditions
Stable and viable economic conditions are prerequisites to measuring the success of any organization. The good economic conditions support the university’ high technological infrastructure by expanding the network, which requires the universities to invest heavily in automated and advanced technologies. The current vision 2030 has been designed to implement the technology in all sectors of the UAE. Under this agreement, several infrastructure projects valued at billions of rupees are under process. These projects, once completed, boost the economy beyond the projections. During the interviews, the participants state that:
“Economic conditions of a country are the best indicators about the progress of a country. Suppose the economy moves in positive directions, then all the connected components of the economy by HEIs. In that case, it will atomically move towards the right direction.”
Law and Order Situation
Just like the economic conditions, the law and order situation of the country also affects all spheres of the economy. All policies to uplift the economy, whether monetary policy, fiscal policy, foreign investment, or tax-friendly regimes, will only render the required results once the security situations of the country are better. In the past, deteriorating law and order situation. During the interview, one respondent commented:
“One and the most important factor of success in law and order situation of the UAE was one of the peaceful and giant technological state of the world but later on the deteriorating security situation impacted it badly similarly success of technology implementation is also dependent on prevailing security conditions.”
It is very much clear from the above statement that the law and order situation of the state is of utmost importance for the success of the technology implementation in UAE universities.
Stakeholders Stance
The stakeholders also indirectly affect to whom they are attached. These stakeholders are the students, teachers and administrators in the education sector. The universities, while transforming any change, have to consider the stance of a shareholder because they affect the university with their voting power. Similarly, the universities should take prior approval from technology assurance companies regarding any transformation of the technology implementation. The vendors who provide these system solutions also affect the performance of technology implementation and success. It concludes the statement to increase and enhance the performance of the technology implementation. Universities should pay proper attention to the pressure of their competitors and shareholders, meet all the regulatory requirements of the stakeholders, and ensure the selection of a system vendor from the market with a good reputation and international standing.
Individual Factors
Attitude
The faculty members’ attitude about the system’s replacement also matters in measuring its technology success. The universities should not consider that once the successful rollout of technology implementation is completed, the target is achieved; instead, they should closely monitor the attitude of their students and teachers to ensure the growth of technology implementation and success. They should remember that the success level is adversely affected if the faculty refuses to accept the technology implementation and feels dissatisfied. The findings of this study highlighted the importance of attitude toward system change, as during the interview, one respondent commented.
“Change is the constant factor in the universe, and it is resisted frequently, and where you feel that your interest is at stake, you become a bitter opponent of this change. I feel how user-friendly technology you are going to launch; it will render a little success till the attitude of the faculty who is using it.”
Interest
There are two factors for measuring the success of any change; one is the fear of failure, and the other is taking an interest in the change. The same is true about the success of technology implementation in UAE universities. The universities should closely monitor whether the students are interested in different features of technology implementation because it plays a vital role in ensuring the success level of technology implementation. During the interview, one responded replied:
“Interest of individual to become part of this change will also be fruitful. Suppose, one takes an interest in diversity or change of technology implementation. In that case, it will help the incumbent move forward because it is the only technology implementation compatible in the market and meets all the requirements of the technology implementation. So, I feel the success of technology implementation increases with the increase of faculty interest.”
Knowledge and Skills
Technical knowledge and professional skills of end-users also play a decisive role in accelerating the success level of technology implementation. All the technologies are user friendly, but they require appropriate knowledge and skills from the staff to render the desired results. Without the required knowledge and skills, the faculty members depend heavily on other colleagues, which eventually suffer their performance. Since the new leader should be hired in these universities have added knowledge and skills, therefore they contribute effectively toward the success level of technology implementation, as stated by one responded during the interview:
“The well-educated staff have more knowledge and skills to use the technology implementation more effectively and will also help others to use the system. So, I consider that proper knowledge and skills are a major factor which increased success of technology implementation.”
Organizational Factors
Culture
The university culture is the sum of norms, practices, shared values, principles, and traditions usually adopted during routine educational activities. The cultural values influence the success level of technology implementation in the UAE universities. The flexible and supportive culture increases the technology implementation success level, whereas the rigid culture negatively affects the system performance. The faculty members in universities relax in open culture to discuss their issues with the higher management for resolution. One respondent has commented:
“You can, first of all, take an impression of any organization from its culture. If the culture is attractive and supportive to the students, then it will positively affect the success of the technology implementation.”
Knowledge Sharing
As there is always room for improvement, sharing the ideas and knowledge between experienced and less experienced faculty members within the university enhances learning abilities, resolves working issues, and promotes mutual understanding. These shared ideas and knowledge ultimately increase the success level of technology implementation because it is also an information system; the more sharing of information ensures more effective use of the technology implementation. During the interview, one participant:
“As goes the maxim if you possess one idea and I also possess one idea, and if we share these ideas, then each of us has two ideas. Similarly, the success of technology implementation depends upon knowledge sharing between the staff working on technology implementation.”
Professional IT Staff
The technology implementation is very technical and requires trained IT staff to guide and teach the end-users. The educational institutions have set up separate IT departments to ensure technology implementation smooth functioning. This department is also responsible for creating user rights and resolution of issue logs within a stipulated period. It also provides that technology implementation is up-to-date and all the ongoing changes have been incorporated. In case of emergencies, this staff acts promptly to fix the problem. Therefore, the role of professional IT staff cannot be overemphasized in measuring the success level of technology implementation as, during the interview, one responded viewed:
“I think that professional staff of IT also play an important role in the success of technology implementation and its implementation because if there is any flaw in technology implementation, then this trained staff can solve the problem enabling the smooth and proper working of technology implementation.”
Staff Training
The management of the universities adopts a variety of training channels such as training sessions, seminars, workshops, and distance training techniques through multimedia to enhance the capacity building measures of the staff. The management should devise separate training programs for the per-implementation and post-implementation phases. The trained staff helps other colleagues in their daily routine and serves the students without difficulty. In UAE, universities invest heavily in training the staff with requisite knowledge and skills. The empirical results also support that training increased the success level of technology implementation in these institutions as one participant said:
“I think that staff should be trained frequently about the system especially in case of technology implementation, a one-day training session in a month should be arranged so that the end-users can apply the innovation in the system. It can increase the success level of technology implementation to a great extent.”
Technical Factors
Implementation Stage
Core transformation passes through three different stages such as pre-implementation stage, transactional phase and post-implementation stage. However, the current study has focused only on the factors affecting the success of technology implementation in the post-implementation stage in the UAS universities. The success level of one-step is positively linked with other educational settings. The success of post-implementation largely depends on the implementation stage as one respondent argued:
“Success of technology implementation depends on the implementation stage as mere devising the sophisticated policies by the organization is not fruitful till the proper execution and implementation of these policies and this implementation should be meaningful in true letter and spirit”
Technical Resources
Technical resources play a very important role in measuring the post-implementation stage of technology implementation. These technical resources, such as double network connection, sufficient stock of systems, ups, generators, printers, etc., are considered mandatory for the smooth working of technology implementation. Technology is ever-changing in the market nowadays, therefore to compete with the competitor, the universities should place enough funds to ensure consistent growth of technology implementation. Since the performance of technology implementation is linked with the availability of electricity, the universities should focus on alternative energy such as wind or solar energy resources because the situation of electricity remained worse in the past. One responded observed that:
“The university in which I am working has arranged two network connections; one is based on landline whereas the second is satellite-based which starts working in case the landline connection is down. Our university is currently using different energy storages as backups, but I think that it should be changed to the solar system to reduce energy cost.”
Technology implementation success in UAE universities is deeply intertwined with a myriad of external, individual, organizational, and technical factors. Economic stability bolsters technological investments, whereas a robust law and order situation provides a conducive environment for technology adoption. Stakeholders, particularly students, teachers, and administrators, play a pivotal role in determining the course of technology transformation. Individually, the faculty’s attitudes, interests, knowledge, and skills largely dictate the efficacy of technology implementation. Organizational culture, the emphasis on knowledge sharing, the proficiency of the IT staff, and regular staff training significantly influence the rate and quality of technology assimilation. Technical aspects, like the rigorousness of the implementation stage and the availability of technical resources, are indispensable for ensuring long-term success. For technology initiatives to thrive in the academic realm, a harmonious alignment of economic, sociopolitical, individual, organizational, and technical elements is imperative.
Discussion
The study examined the success of technology implementation in 27 UAE universities using a sequential explanatory research design. Based on quantitative findings, face-to-face semi-structured interviews were conducted to find various factors affecting the success of technology implementation through the qualitative method. Hypotheses 1 empirically established that system quality significantly influenced individual (faculty) performance. These research findings are consistent with empirical studies (Bharati & Chaudhury, 2006; Cidral et al., 2018; Kositanurit et al., 2006; Shih, 2004). These empirical studies provide enough theoretical and empirical background to believe that the technology implementation system enhances individual performance. The universities equipped with the latest and fully automated technology are reaping the benefits of increased UAE University’s faculty, operational, and university performance (Cidral et al., 2018). By improving the accuracy, consistent use of technology implementation, system reliability, efficiency and ease of use of technology implementation, these universities boost academic’ performance. Similarly, qualitative findings established that the more customization of technology implementation, the more technology implementation success would be.
The empirical results of hypothesis 2 established that information quality significantly influenced faculty performance. Thus, the results are consistent with other studies (Bharati & Chaudhury, 2006; Kositanurit et al., 2006; Mtebe & Raisamo, 2014. It was stated to reap technology benefits, universities should continuously customize their technology implementation. From the information perspective, this customization is in the shapes of various reports formatting, graphing, table and charts to understand them easily. I also conclude the results of hypothesis 3 with a significant and positive impact of service quality on faculty performance. The findings are aligned with the previous studies (Alkhalaf et al., 2021; Almutairi, 2005; Halawi et al., 2007; McGill & Klobas, 2005). The more consistent, reliable, response-oriented, assured, tangible and compassionate services of the computer departments are, the faculty performance increases in UAE universities. Similarly, the qualitative findings claimed that proper and timely availability of technical resources and IT professionals increases technology implementation success in UAE universities. The readily available technical resources such as wireless backup connections, users’ systems, and relevant IT devices are not only necessary for technology implementation to end-users, but they also assist the computer department in providing active support to faculty and students in emergencies.
Hypothesis 4 also established a strong significant impact of faculty performance on operational performance. The results endorsed the study’s findings by Grilley (2004), who concluded that faculty performance had positively influenced operational performance. Since there are individuals who streamline all the technology implementation operations, universities should pay great attention to their academic performance to obtain good results from university operations. The qualitative phase concludes that the faculty’s motivational tools, such as incentives, regular training, decision-making involvement, and assigning appropriate leadership roles, inspire the faculty members and increase their performance. Hypotheses 5 empirically established a positive impact of operational performance on organizational (university) performance, proving that automated, effective and efficient technology implementation operations increase the performance of the universities by increasing productivity and turnover and minimizing operational cost. The results are consistent with the findings of previous studies (Hyvönen, 2007; Duarte, 2011). The empirical results recommended that universities increase their operational performance by continuously updating automated solutions and relevant IT infrastructure to increase university performance. From the qualitative perspective, coordination between departments improves the performance of technology operations, which resultantly enhances university performance.
Hypothesis 6 demonstrated a significant impact of faculty performance on university performance. The results endorsed the findings of past studies (Bharati & Chaudhury, 2006; Kositanurit et al., 2006; Lee & Lee, 2012). The findings suggested that UAE universities should increase the faculty performance of professors, eventually enhancing university performance. The section of the qualitative conclusions confirmed that the end-users perceived learning capacity, attitude, professional skills, and knowledge capabilities ultimately contribute to technology implementation success in UAE universities. The study found that organizational culture also contributes to the success of the technology implementation. The universities in UAE should flourish flexible and supportive cultural norms such as realistic rules, unbiased procedures, clearly defined hierarchy, and empowering line managers will eventually achieve the university’s vision and mission.
Contributions and Implications
The current research robustly corroborates and extends the theoretical understanding of technology implementation in UAE higher education. Empirical evidence robustly shows the significant influences of system quality, information quality, and service quality on faculty performance. Further, it elucidates the cascading effect of faculty performance on operational and, ultimately, organizational (university) performance. This establishes a strong theoretical framework emphasizing the interdependencies of quality facets and their cumulative impact on institutional success in the UAE academic context. The results contributed to the success of information system research in the following ways: Firstly, this study validated the model of technology implementation success at multiple levels, that is, individual (faculty) level, operational level and organizational (university) level. Secondly, the study extended the D&M IS success model by incorporating the “operational performance” relevant to evaluate technology implementation success at the operational level. The extended model was validated in UAE universities. Thirdly, the study investigated the technology implementation success in UAE universities and found a significant and positive relationship among faculty, operational and university performance that was overlooked in earlier IS success evaluation models. Fourthly, various critical factors affect technology’s success in the education sector. These factors were divided into four categories, that is, external, individual (faculty-related), organizational (university-related), and technical. The findings from these contextual factors helped us understand the effects of both external and internal factors on the success level of technology implementation.
Practical Implications
This study outlined important research implications for future practice. HEIs in UAE efficiently use technological facilities to increase their profitability and student performance. Technology implementation and usage at a large scale require the researcher to provide IS model to evaluate the post-implementation success of the technology implementation and identify various contextual factors that affect the technology success level. Most studies have only examined the effects of technology implementation on UAE universities. The current research touched the unexplored area by providing a validated model for evaluating the post-implementation success of technology implementation at individual (faculty), operational and organizational (university) level in UAE universities. The findings of the study suggest that the university management should focus on improving service quality, system quality, and information quality to enhance individual faculty performance, which, in turn, would improve operational and organizational performance. Additionally, the management should recognize and reward the performance of individual faculty members to motivate and encourage them to perform better.
Limitation of the Study
The current study has some limitations and future directions such as the present study only evaluated IS model with six dimensions, that is, system quality, information quality, service quality, individual performance, operational performance and organizational performance. Future research may employ more complex modes to measure the success of technology implementation by incorporating more dimensions such as technology implementation, adaptability and vendor. The study had only examined the impacts of the technology implementation at individual (faculty), operational, and organizational (university) levels but ignored other classes, such as the student level, directly influenced by technology implementation and performance. The study used a convenient sampling approach to collect the data from public universities, which has the issue of generalizability of the findings to the other private universities. The study includes the limitations and future directions such as:
The study did not consider external factors like economic conditions, law and order, political conditions, stakeholders’ stances, and energy infrastructure.
The field surveys focused solely on universities and overlooked other integral players within these institutions, like different departments.
Expand the research to consider the impacts of external factors on service quality, including economic conditions, political climate, and energy infrastructure, among others.
Incorporate diverse stakeholders within universities, such as various departments, to gain a more comprehensive understanding.
Extend the study to evaluate the success of technology implementation in other sectors, including textile, manufacturing, and hospitality.
Conclusion
The study examined technology implementation in UAE universities, revealing significant findings in line with past studies. Factors such as system quality, information quality, and service quality greatly influence faculty performance, ultimately influencing operational and university performance. Furthermore, technology customization, consistent service quality, and inter-departmental coordination were key determinants of technology implementation success. In addition, faculty motivation and participation, continuous system updates, and an adaptable organizational culture substantially contribute to university performance. Recent literature, including works by Camilleri (2021), Riandi et al. (2021), Jameel et al. (2021), Ramírez-Hurtado et al. (2021), and Sewandono et al. (2023), align the imperative of high-quality e-learning services and the consequential influence on student and faculty satisfaction, especially in a post-COVID-19 academic landscape. The implementation of technology, service quality, and user satisfaction has proven pivotal for academic excellence and efficiency in the current era of higher education.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Data Availability Statement
The dataset is with Rabdan Academy, UAE so I can provide on demand