Aligning practitioner's perception: Empowering MAST framework for evaluating telemedicine services

Abstract

Objective

Telemedicine is a digital substitute for in-person healthcare service delivery systems that has gained popularity amid the global COVID-19 pandemic. The objective of this study was to evaluate telemedicine compatibility from the perspective of healthcare practitioners to enhance the effectiveness and spectrum of the Model for Assessment of Telemedicine.

Method

Primary and Secondary Healthcare and King Edward Medical University extended their respective telemedicine services in 2020 where 24,516 patients were benefited from the telemedicine services provided by 1273 doctors from different specializations. A cross-sectional survey via online questionnaire was conducted among purposively sampled 248 healthcare practitioners designated at telemedicine portals in the public sector; further analysed by descriptive analysis and Monte Carlo Feature Selection.

Results

Healthcare practitioner perception was analysed explicitly and found significant in addition to the existing domains under multidisciplinary assessment in the Model for Assessment of Telemedicine model. The variables of subdomains integration with healthcare system, patient facilitation, technology ease, capacity building, ethical integrity, outcome assessment and communication gap under proposed healthcare practitioner perception domain were found interdependent. The variables of patient satisfaction, resource preservation, healthcare practitioner satisfaction, digital connectivity, user-friendliness, and patient safety were found to be of higher importance (RI values). However, the compatibility of telemedicine with the healthcare system was also influenced by interdependencies (RI plot) and multifaceted interactions of variables derived from the healthcare practitioner perception.

Conclusion

The variables of healthcare practitioner perception were exhibiting various weightages of importance and interdependencies in determining the compatibility of telemedicine within the healthcare system and recommended to be considered in the Model for Assessment of Telemedicine framework.

Keywords

Health eHealth telemedicine telehealth digital health health informatics

Key questions

What is already known?

Researchers have reported a gradually increasing trend of patient satisfaction with telemedicine services in various frameworks; however, most of the surveys were conducted on telemedicine services for a particular disease or specialty. It has also been established that patients can save time and travel expenses with the help of telemedicine services. Researchers have also outlined the requirements for infrastructure for running virtual healthcare services flawlessly under specific conditions.

What are the new findings?

HCP perception is recommended as an additional domain under multidisciplinary assessment in the MAST model. Monte Carlo feature selection and RI plot methods have been utilized to identify significant variables and interdependencies for determining the variables of importance for enhancing the effectiveness of telemedicine services. The effectiveness of current telemedicine and healthcare services in Pakistan can be improved by addressing the significant and interdependent variables of various weightings. Telemedicine can reduce the burden on healthcare facilities, particularly in developing countries, due to the scarcity of human, capital, and infrastructure resources in healthcare facilities.

What do the new findings imply?

The results of this study can be used in similar healthcare settings of other developing countries with thriving resources in healthcare delivery systems. The addition of HCP perception to the MAST framework can lead to further conscious research and decision-making. The development of the guidelines for telemedicine services execution is the need of hour for ensuring effectiveness and quality of services. This research will help the healthcare researchers to develop further evidence-based studies and recommendations to enhance the effectiveness of telemedicine services in healthcare delivery systems. In addition, this study will help policy makers and health strategists to plan effectively in the healthcare sector.

Introduction

Telemedicine is an emerging field in medicine and health care.¹ Globally, there has been a shift in the doctor‒patient interaction model, which may reduce both cost and time.² The impact of health care via telemedicine in developing countries is greater than that in developed countries.³ During the COVID-19 pandemic, telemedicine was widely adopted across the globe, as resources and human resource capacity were on the verge of limit exhaustion. Home recovery programmes and virtual teleconsultations were chosen to ease the strain and stress on the healthcare system during pandemic.^4,5 In impoverished nations such as Pakistan, whose healthcare services face limited resources and are overburdened, their ability to respond to emerging pandemics is severely compromised. The provincial Punjab Government in Pakistan successfully established telemedicine clinics to reduce hospital burdens during the pandemic COVID-19 virus.

Multiple theories,⁶ such as health belief model (HBF),⁷ technology acceptance model (TAM),^8,9 diffusion innovation theory (DIT),⁸ technology task fit (TTF),¹⁰ model for assessment of telemedicine (MAST),¹¹ preferred reporting items for systematic reviews for systematic meta-analysis (PRISMA),¹² theory of planned behaviour (TPB),¹³ unified theory of acceptance and use of technology (UTAUT),¹³ social cognitive theory (SCT)¹⁴ and technology-organizational-behavioural theory (TOE),¹⁵ have been used to determine the acceptance and effectiveness of telemedicine. Various questionnaires, such as telehealth usability questionnaire (TUQ), telemedicine satisfaction questionnaire (TSQ), service user technology acceptability questionnaire (SUTAQ), client satisfaction questionnaire (CSQ), questionnaire for user interaction satisfaction (QUIS), system usability scale (SUS), patient satisfaction questionnaire (PSQ), and technology acceptance model (TAM) are being used to assess multiple aspects of telemedicine.^16,17 The HBF focuses on cognitive constructs which include perceived susceptibility, severity, benefits, and barriers; however, it does not consider emotional, social and cultural aspects which may affect health behaviour.¹⁸ HBM has a low predictive value (20% to 40%) in explaining health behaviours compared to other models or theories.^19,20 TAM is easier and inexpensive to apply and focuses on intention towards use however it provides general information about ease of usefulness.¹⁰ TTF models adopt a very logical stance by presuming that users, regardless of their attitude toward the IT, choose to utilize it when it offers advantages like better job performance. TTF does not focus on social efforts influencing technology and effort in using technology.¹⁰ By incorporating eight different models into one, UTAT is the most comprehensive acceptance model, however, consideration of more external factors in different technology area is required in this model.¹⁰ The SCT argues that learning in humans is based on observation of other people's behaviour. According to critiques it is not a unified theory, and the concept of reciprocal determinism is not well-organized too.²¹

The Model for Assessment of Telemedicine (MAST) is a recommended model for the evaluation of telemedicine projects by the European Commission and World Health Organization (WHO) that covers seven domains under multidisciplinary assessment, namely, health problems and characteristics of the application, safety, clinical effectiveness, patient perspectives, economic aspects, organizational aspects, and sociocultural, ethical, and legal aspects.²² MAST is based on the European network for health technology assessment (EUnetHTA) core model and has three steps: preceding considerations, multidisciplinary assessment, and transferability assessment.²³ The multidisciplinary assessment included further sub-domains as health problems, safety, clinical effectiveness, patient perspective, economic aspects, organizational aspects and socio-cultural, ethical and legal aspects.²³ The systematic analysis revealed that assessment of organizational aspects, clinical effectiveness was mostly based on experience and perception of healthcare practitioner however economic aspect is least discussed in most of the studies.²³ MAST is the most widely used model for telemedicine projects.^24,25

Monte Carlo Feature Selection is an advanced machine learning technique which consists of three steps, i.e. screening, ranking, and selecting. The advantage of this advanced technique is to quickly highlight the most significant key features, easily determine smaller yet important features and bring more accurate results with simpler explanation of complex models and interdependencies.²⁶ MCFS is a powerful tool for feature selection with the main advantages of robustness (reliable ranking of features by using multiple random samples and constructing tree classifiers), scalability (adaptive large dataset) and flexibility (applicable to both continuous and categorical data). MCFS is recommended to be used in scenarios where data is noisy, has complex interdependencies, pre-processing for classification and ranking of the features’ importance, proving insights into which feature influences dataset. Some alternative approaches other than MCFS are filter, wrapper, embedded, hybrid and meta-heuristic methods. The competitive advantage of MCFS over other methods is its handling of high-dimensional data, robustness due to noise, capability to capture non-linear interactions, comprehensive feature ranking, and versatility.²⁷

Broadly, MCFS is known to be used in the fields of bioinformatics, text mining, financial data analysis, medical imaging, environmental science, telehealth, engineering and manufacturing. Monte Carlo Feature Selection (MCFS) has promising applications in telemedicine, particularly in enhancing the accuracy and efficiency of predictive models used in remote healthcare. It has proven as a promising technique in the field of telemedicine where it has been used for disease prediction and diagnosis, personalized treatment plans, remote monitoring and management, and enhancing telehealth platforms.²⁷ Although Monte Carlo feature selection provides a probabilistic and flexible approach, in high-dimensional environments it might not be as stable and efficient as other well-established methods.²⁸ Compared to deterministic approaches like recursive feature elimination (RFE) or regularization techniques (e.g. Lasso), MCFS is computationally intensive and slow, especially with large datasets or high-dimensional feature spaces. The randomness in MCFS may lead to unstable results as different selected features are exhibited in different runs, in contrast to tree-based models (e.g. Random Forest) which may provide more stable feature importance measures.²⁷

The utility of telemedicine depends on its perceived usefulness by healthcare practitioners.²⁹ A meta-analysis suggested that physician satisfaction increases when physicians are involved in the development of telemedicine software.³⁰ Healthcare practitioners are the gatekeepers for telemedicine because patients use such innovative services if they are given confidence from their healthcare providers about such innovations. The utilization of services depends on the willingness of the service provider to access telemedicine; thus, these services play a major role in the diffusion of telemedicine services among the community.³¹ To our knowledge, no such thorough study employing the MAST framework with HCP perception has been published to assess telemedicine compatibility in the healthcare system. This study is designed to determine the compatibility of telemedicine in the healthcare systems of underdeveloped nations such as Pakistan by filling the vacuum in the literature by concentrating on the crucial component of HCP perception integrated within the MAST model.

Methods

Study site

In Pakistan, the healthcare delivery system is classified into two tiers: primary and secondary healthcare (PSHC), and tertiary healthcare. PSHC comprises of Basic Health Units (BHU), Tehsil Headquarter (THQ) and District Headquarter (DHQ) hospitals while tertiary healthcare covers specialized healthcare hospitals and associated medical universities. Lahore is the capital of Punjab Province in Pakistan and has 51 public sector tertiary care hospitals, receiving maximum referrals from all cities in the province. The Lahore Division was selected for study purposes because it was declared an epicentre of COVID-19 in March 2020. The HCPs identified in telemedicine clinics providing synchronous telehealth services at BHU (primary and secondary healthcare) and KEMU (tertiary healthcare) in Lahore were included in the study. The telemedicine services were extended in March 2020, and data for the subsequent 3 years was analysed.

Study design

This analytical cross-sectional study was conducted to integrate the perspective of healthcare practitioners for enhancing the compatibility of telemedicine with the prevailing healthcare delivery system.

Data collection instrument

In this research, HCP questionnaire (Supplemental File 1) based on the TUQ was developed and modified according to the study objectives and prevalent socio-cultural norms, considering its compatibility with the healthcare system of Pakistan.^{16,17,29,32–37} The formulated questionnaire was encompassing 24 variables and was pilot tested among HCPs. A valid questionnaire with a Cronbach's alpha of 0.87 was used in this study. HCP perspective was analysed and proposed as an additional important domain under the multidisciplinary assessment of the MAST framework.

Variables studied

The proposed domain of HCP perception is further classified into eight subdomains: HCP demographic analysis (DA), integration with healthcare system (IHS), patient facilitation (PF), technology ease (TE), capacity building and guideline requirement (CB), ethical integrity (EI), outcome assessment (OA) and communication gap (CG). The dependent variable was compatibility with the healthcare system (V5), which was studied with various variables in the subdomains of HCP perception in supplemental file 2.

Sample, sampling size and sampling technique

Training was provided to a total of 1797 health care practitioners at the PSHC and KEMU tiers. Telemedicine consultation services to a total of 24,605 patients were provided by 155 practitioners at the PSHC and 1118 practitioners at the KEMU. The ethics approval by the respective committees was sought and consent from the participants was taken prior to the execution of the study, details of which are shared in the declaration section. All participants gave written consent to participate in the study. Furthermore, a digital consent was also incorporated in the questionnaire preceding the questions. The questionnaire was only submitted if accompanied with the acknowledged digital consent of the participant for participation in the survey including the purpose of survey, risks and benefits, assurance of anonymity, etc.

The practitioners who performed more than nine teleconsultations in designated public sector health facilities and were willing to participate in survey were eligible as per inclusion criteria and those who performed less than ten teleconsultations or were not willing or provided consent to participate in survey were excluded from study as per exclusion criteria. As an eligibility criterion, only those willing practitioners who had conducted more than nine telemedicine consultations were found eligible. The sampling technique for the selection of healthcare providers was purposive sampling to ensure maximum participation from all specialities with 95% confidence interval (CI). A total of 401 practitioners were outreached, 248 of whom willingly participated and responded in a timely manner to the survey. The collective minimum sample size³⁸ was calculated as 197; however, to increase the accuracy, a total of 248 surveys comprising 172 responses from KEMU and 76 responses from PSHC were included as shown in Figure 1.

Figure 1.

Flow chart of sample selection of HCP for survey.

Data collection procedure

An online survey was conducted from 248 healthcare practitioners to assess the effectiveness of telemedicine services and responses were received in time with consent of participation in the study.

Data processing and analysis

Monte Carlo Feature Selection and RI plot methods in R software were used to determine the weightages of variables and their interdependencies. The number of interactions were 23 with a single dependent variable to establish interdependencies. Healthcare practitioner perception was analysed and proposed for inclusion in MAST model, considering that assessment of adaptability of intervention is inefficient without inclusion of service-care provider insight.

Patient and public involvement

No patient or public involvement took place in the design or conduct of this study. We involved healthcare practitioners who were extending their services at the telemedicine clinics. The perception of health practitioners was analysed through a consented survey comprising of questions related to various aspects of telemedicine services.

Results

The demographic analysis of the study participants in terms of age group, gender, specialty, and level of healthcare tier served is shown in Table 1.

Table 1.

Demographic distribution of healthcare practitioners.

Health facility (HF) tier
PSHC	76 (30.64%)
KEMU	172 (69.36%)
Healthcare practitioners
Specialists	219 (88.31%)
General practitioners	29 (11.69%)
Age of healthcare practitioners
Aged less than 40	133 (53.63%)
Aged 40 or above	115 (46.37%)
Gender of healthcare practitioners
Female	105 (42.34%)
Male	143 (57.66%)
Specialty
Medicine	59 (23.79%)
Paediatrics	41 (16.53%)
Dermatology	23 (9.27%)
Obstetrics and gynaecology	37 (14.92%)
General practitioner	24 (9.68%)
ENT	14 (5.65%)
Surgery	13 (5.24%)
Cardiology	11 (4.44%)
Orthopaedics	9 (3.63%)
Psychiatry	7 (2.82%)
Nutrition	5 (2.02%)
Pulmonology	4 (1.61%)
Ophthalmology	1 (0.40%)

The calculated sample size for eligible HCPs was 197.³⁸ The eligible 401 HCPs (KEMU n = 308, PSHC n = 93) were outreached for online survey. A total of 62% (n = 248) HCPs responded. The online survey involved consent for participation. Only those participants could proceed for submission of responses who agreed to the consent for participation. No missing data was reported in this study because incomplete questionnaire was not allowed to be submitted by the system Figure 1. An online survey comprising 20 questions based on the research objectives was conducted. The analysis of the HCP responses on a Likert scale is shown in Figure 2.

Figure 2.

Analysis of HCP responses on Likert scale.

IHS subdomain: About 63% of HCPs considered telemedicine compatible with the current healthcare system. With respect to the requirement of telemedicine as an integral part of healthcare facilities, 54.03% of the healthcare practitioners strongly agreed, 29.44% agreed and 16.53% disagreed. Regarding the requirement of telemedicine in pandemic, 33.06% strongly agreed, and 66.94% agreed. Approximately 62% of the healthcare practitioners were satisfied with the telemedicine services. About 23.79% HCP recommended telemedicine services for 24 h, 21.77% recommended in morning and evening, 41.53% recommended in morning only while 10.08% recommended in evening only.

PF subdomain: For the question regarding HCP perception of patient comfort, about 74% responses were surveyed as satisfied, whereas 53% of the HCPs were found to be satisfied for perceived patient satisfaction.

TE subdomain: The questions relevant to healthcare practitioners’ perceptions of the quality of digital connectivity provided responses of 45.97% as satisfied, 33.06% as average and 20.97% as not satisfied. When the user friendliness of the telemedicine portal was questioned, 51.61% as satisfied, 38.31% as average and 10.08% as not satisfied.

CB subdomain: The requirement for capacity building for optimum utilization of telemedicine portals was strongly agreed by 59.27%, agreed by 5.65%, disagreed by 31.05% and strongly disagreed by 4.03% of HCPs. However, 60.89% of the respondents agreed on the inclusion of telemedicine in medical education programmes. The requirements for quality healthcare standards were agreed by 82.66% of HCPs.

EI subdomain: The patient safety was agreed by 77.02% of HCPs. When satisfaction with patient privacy at the telemedicine clinic was assessed, 70.36% were satisfied.

CG subdomain: When healthcare practitioners’ perceptions of the doctor‒patient communication barrier were assessed, 9.68% strongly agreed, 40.32% agreed, 47.18% disagreed and 2.42% strongly disagreed. The most convenient age-group and gender-based group for utilization of telemedicine services in the opinion of HCPs based on their interaction with patients was enquired. The patient gender-based communication convenience was 51.21% for both males and females, 27.02% for males only and 21.77% for females only, whereas the patient age-group-based communication convenience was 4.84% for 10–19 years, 62.10% for 20–34 years, 27.02% for 35–49 years and 6.05% for 50 years and above.

OA subdomain: Approximately, 83.47% of HCPs acknowledged reduced number of physical visit of patients. A total of 76.61% of the practitioners agreed with the idea that telemedicine can help in resource preservation. When asked about the most impactful outcome of telemedicine in healthcare, approximately 44.35% of the voted for reduction of burden on outpatient departments, 20.97% believed for early diagnosis and treatment of patients, 16.13% opted for reduction of the financial cost at the patient end, and 18.15% opted for the reduction of referrals from rural areas.

MCFS method reflects the importance of various variables to the dependent variable, arranged in descending order of their RI values highlighting various weights within inter and intra-subdomains, thus revealing a complex network of interdependencies exhibiting compound influence on the outcome variable as shown in Figure 3(a)–(b). The interdependency matrix is an amplification of the interdependency of individual variables with other variables among various subdomains of HCP perception as shown in Figure 3(c).

Figure 3.

Monte Carlo feature selection and interdependency plot interpretation: (a) variables importance (RI value), (b) interdependency (RI plot), (c) interdependency matrix of variables and subdomains of HCP perception.

The variables that exhibited strong importance (high RI values) are patient satisfaction (V12), resource preservation (V15), HCP satisfaction (V14), health facility tier (V1) and digital connectivity (V6). The other variables showed moderate to weak importance while the specialty of HCP (V4) was found to be of the weakest importance and can be classified as not an important variable as depicted in Figure 3(a). Figure 3(b) displayed dense interdependencies among various variables through RI Plot. The variables with most displayed interdependencies are found to be digital connectivity (V6), user-friendliness (V7), HCP satisfaction (V14), resource preservation (V15), patient satisfaction (V12), capacity building (V20), patient safety (V10) and healthcare facility tier (V1).

The Figure 3(c) represents interdependency of various variables among eight subdomains of HCP perception including outcome acceptance (OA), integration with healthcare system (IHS), patient facilitation (PF), technology ease (TE), capacity building and guideline requirement (CB), ethical integrity (EI), demographic analysis (DA) and communication gap (CG) which is further elaborated in Figure 4. The subdomains of HCP perception in Figure 4 represent variables in descending order of importance (RI value) within each subdomain against dependent variable V5 compatibility with healthcare system. The strong, moderate, and weak interdependencies within each subdomain are also highlighted in Figure 4 along with elaboration of detailed interdependencies among variables in Figure 3(c).

Figure 4.

Subdomain variables analysis of healthcare practitioner (HCP) perception.

The TE (V6 digital connectivity) exhibited interdependency with the CB (V20 capacity building, V9 quality standards), TE (V7 user-friendliness), IHC (V14 HCP satisfaction, V18 integral part of healthcare facilities), PF (V12 patient satisfaction), and CG (V22 communication barrier, V23 age-based patient convenience). Similarly, the TE (V7 user-friendliness) represents interdependency with the DA (V2 HCP age group), CB (V20 capacity building), EI (V10 patient privacy) and OA (V11 impact). The EI (V8 patient privacy) is interdependent on the PF (V13 patient comfort), CB (V20 capacity building), OA (V11 impact) and TE (V7 user-friendliness). The EI (V10 patient safety) is found to be interdependent on the TE (V7 user-friendliness), OA (V11 impact), CB (V20 capacity building) and CG (V24 gender-based patient convenience).

PF (V12 patient satisfaction) having highest RI value (0.05462) is interdependent on TE (V6 digital connectivity), IHC (V14 HCP satisfaction) and OA (V15 resource preservation). Age-based patient convenience (RI 0.01341) was found to be more important and interdependent than the other two variables in the subdomain communication gap (CG). The V22 communication barrier is interdependent on IHC (V19 recommended duration) and OA (V11 impact, V17 reduced visits). The capacity building requirement (RI 0.1917) has greater importance than the quality standard requirement (RI 0.1854). The variables of subdomain capacity building (CB) are interdependent not only on each other but also on other subdomains of HCP perception. In addition, the CB (V20 capacity building) is interdependent on the EI (V8 patient privacy, V10 patient safety), DA (V1 health facility tier, V2 HCP age group), CG (V23 age-based patient convenience), IHC (V14 HCP satisfaction), OA (V11 impact), and TE (V6 digital connectivity, V7 user-friendliness). CB (V9 quality standards) is interdependent with OA (V11 impact, V15 resource preservation) and TE (V6 digital connectivity). The CB (V21 telemedicine requirement in medical education) was found to have very low importance (RI 0.00897) with no visible interdependency.

The variables of outcome analysis (OA) represent greater significance and interdependency. The OA (V11 impact) is interdependent on the TE (V7 user-friendliness), CB (V9 quality standards, V20 capacity building), IHC (V19 recommended duration), CG (V22 communication barrier) and EI (V8 patient privacy, V10 patient safety). The OA (V15 resource preservation), with the second highest RI value (0.5131), was interdependent on DA (V1 HF tier), TE (V6 digital connectivity), CB (V9 quality standards), IHC (V14 HCP satisfaction, V18 integral part of HF, V19 recommended duration), and PF (V13 patient comfort, V15 patient satisfaction). The OA (V17 reduced visits) exhibited interdependencies with the PF (V13 Patient comfort), IHC (V14 HCP satisfaction, V19 recommended duration), OA (V15 resource preservation) and CG (V22 communication barrier). Similarly, IHC (V14 HCP satisfaction) is interdependent on OA (V17 reduced visits), CB (V20 capacity building), and TE (V6 digital connectivity); IHC (V16 requirement in emergencies) is interdependent on PF (V13 patient comfort); and IHC (V18 integral part of HF) is interdependent on OA (V15 resource preservation) and TE (V6 digital connectivity).

Discussion

In the technical ease (TE) subdomain, digital connectivity (RI 0.02805) is more important than user-friendliness because of its higher RI value. The digital connectivity and user-friendliness of telemedicine services variables exhibit strong interdependency with each other as well as with the variables of the other subdomains. This finding infers that user friendliness and digital connectivity influence HCP and patient satisfaction. The interdependency of the variables of technology ease subdomain with the variables of the capacity building subdomain indicates that the level of technology acceptability is dependent on capacity building and quality standardization of services. The successful integration of telemedicine technology in the health care system and the enhancement of its impact are based on user satisfaction, which requires measures in terms of addressing communication gaps, ethical concerns, provision of continuous capacity building and technology support. The problems faced by HCP in a study conducted in India were communication barrier (55.7%), poor network connectivity (34%), issues related to diagnosis and investigations (32%), and problems with medicine prescription (2%).³⁹ Kum Fai Yuen states that adoption behaviour depends upon perceived value, emotion, and intention; moreover, flawless communication, continuous education, training, and adequate technology exposure are key for smooth influx of technology in any healthcare system.⁴ A detailed study in China involving 1349 practitioners highlighted the issues of incomplete digital systems for telemedicine and long waiting times for patients; however, convenient operating systems have a significant positive impact.⁴⁰ According to a meta-analysis study, technical issues hinder the use of telemedicine,⁴¹ and high-speed, high-quality connections are necessary to support telemedicine consultations without interruption.

Patient safety (RI 0.02044) was found to be of higher importance as compared to patient privacy (RI 0.01999) with almost same interdependencies. A total of 77.02% and 70.36% of the HCPs reported satisfaction with patient safety and privacy, respectively. This infers that considering ethical aspects and imparting relevant training modules to HCPs while incorporating telemedicine technology in the current healthcare system will have both direct and indirect effects on patient trust, convenience, and comfort, which in turn contribute to the outcome. A study conducted in an emergency department showed that the frequency of physician-related medication errors was at least three times greater in patients who did not receive telemedicine consultation than in those who did, providing evidence for a significantly reduced risk of physician-related medication errors.⁴² A deficient technology infrastructure in telemedicine clinics may negatively affect patient treatment and may also increase the workload on physicians’ side by side⁴³; therefore, patient safety risks related to telemedicine services must be addressed with the help of a human factor systems approach.⁴⁴ The implementation of service delivery standards is required to address ethical concerns.⁴⁵

An examination of the patient facilitation (PF) variables revealed that 84.68% and 73.79% of the HCPs reported satisfactory levels of perceived patient comfort and satisfaction respectively, based on their personal interactions with patients. Of all the variables in the study, patient satisfaction (RI 0.05462) had the highest level of significance. HCP believed that telemedicine services were helpful in providing comfort to patients during pandemic by providing effective treatment and advice remotely. Additionally, HCP were of the view that the positive impact on the comfort and contentment of patient is strongly dependent on the effective telemedicine healthcare system and can become as their first choice for consultation. Patient satisfaction and HCP satisfaction were also found to be correlated. In Ethiopia, 52.9% of diabetic patients were satisfied with teleconsultation services.⁴⁶ The impact of telemedicine services can be enhanced if the acceptability belief is disseminated among users by extending support for user-end facilitation and comfort.

Age-based patient convenience has greater significance and interdependencies than the remaining two variables of communication gap (CG) subdomain. The interdependency matrix illustrates that telemedicine services can have a greater overall impact on patient visits, appropriate referrals, early screening and diagnosis by bridging communication barriers. By protecting patient privacy, gender-based issues can be addressed. Training in communication skills helps alleviate age-based limitations. The communication gap was acknowledged by 50% of the HCPs. Among HCPs, 51.21% considered that both males and females were convenient in teleconsultations. Among age-based patient groups, 62.10% of HCP agreed with 20–34 year age group to be more comfortable while accessing telemedicine services. A very small percentage of respondents reported the convenience of young, elderly, and female patients, which indicates that these choices may be dependent on their specialty areas, such as paediatrics, geriatrics or gynaecology. A retrospective study was conducted in Bangladesh, where males were more inclined towards telemedicine than females.⁴⁷ Elderly patients are likely to be less satisfied with this mode of service, and they require help from younger relatives.^48–50 The decline in telemedicine trend is seen with increasing age, the reason for which might be limited computer knowledge, accessibility and adaptability to technology.⁵¹ The telemedicine poses greater doctor‒patient communication barrier because of patient confusion and discomfort while teleconsultation, and in some scenarios, teleconsultations may be more physician dominated session which may further add up to increased communication barrier between physician and patient system.^47,52 According to a meta-analysis, nonmedical behaviours and interrupted eye contact play a negative role in patient satisfaction.³⁰ However, the accessibility of telemedicine is more advantageous than is associated with the communication gap, as it can be overlooked and easily mitigated by proper facilitation of mass and user-friendly integration with the healthcare system.⁴⁷ Mahdi endorses that despite technical developments, challenges of governance and stakeholder support and policy implementation, capacity building and skill development requirement along with deprivation of effective logistics may hinder adaptability to this kind of care of the target population.⁵³

Approximately 61% of the respondents were in favour of the inclusion of telemedicine in medical education. Among HCPs, 64.92% and 82.66% emphasized capacity building and standardization requirements, respectively. Patient privacy, patient safety, quality standards, technical proficiency, impact and use-friendliness are interrelated with capacity growth. Numerous training programmes focused on patient care can improve professionalism and quality of care while removing barriers related to demography, communication, and ethics. Training and self-sufficiency play a key role in overcoming the hesitation of practitioners in telemedicine.^54,55 The training modules constituting ethical perspectives, quality standards and hands-on-training sessions can overcome various communication gaps, reinforcing the impact of telemedicine services in healthcare systems.⁵⁶ In addition, competency-based, and outcome-oriented training workshops on self-efficacy can prepare HCPs for teleconsultation.^57–59 According to a study conducted in Pakistan, a satisfactory attitude was found in 80% of medical students regarding telemedicine services and about 74% of medical students were inclined to use it in future. It was also commented in the study that the dearth of knowledge among medical students is due to a lack of telemedicine regulations and insufficient awareness due to compromised marketing strategies about telemedicine. Incorporation of telemedicine in medical education may enhance the acceptance of telemedicine among future HCPs.⁶⁰

The variables of outcome analysis (OA) represent greater significance and interdependence. Approximately 76.61% and 83.47% of the HCPs believed in resource preservation and minimized visits, respectively, to healthcare facilities. The majority of the HCPs were of opinion that the most impactful outcome of telemedicine services is a reduction in the burden on the outpatient department. The perceived impacts and outcomes of telemedicine, such as a reduced burden on hospitals and resource preservation, are strongly dependent on the level of technical appropriateness, continuous capacity building, addressing ethical considerations, and building trust of healthcare practitioners in terms of their patient comfort, satisfaction, safety, and privacy. Zhang proposed a complete national health system including telemedicine as a fundamental approach for governments to address the inequal distribution of scarce medical resources between urban and rural areas.⁶¹ In another study in China, 92.5% of health care professionals believed that telemedicine aided in the reduction of referrals and more than 95% were willing to continue and recommend teleconsultations.⁶²

Approximately 63% of the HCPs reported telemedicine compatibility with the current healthcare system, and 62% reported satisfaction with the telemedicine initiative. Telemedicine was recommended in all the responses as an essential requirement during pandemics, and most of the HCPs (83.47%) also supported the use of telemedicine as an integral part of healthcare facilities. The interdependence among variables suggests that telemedicine can be successfully integrated into healthcare facilities and that the incorporation of technical support and capacity building programmes will in turn positively influence clinical outcomes, patient facilitation, comfort, and healthcare provider satisfaction. In a study carried out on 272 healthcare providers, 80% endorsed the future use of telemedicine. For example, younger doctors with a mean age of 40 years preferred to use telemedicine in the future, particularly during follow-up visits, during urgent care, and under specific conditions such as behavioural health, dermatology visits, and chronic care management. The main concerns for those not opting were inadequate patient care, lack of physical patient interaction, technology issues, and lack of necessities.⁶³ In contrast, a meta-analysis revealed that telemedicine reduces costs compared to regular care and enables physicians to deliver optimum clinical care.⁶⁴ Telemedicine can ensure the safety of patients and provide effective treatment during public emergencies.⁶¹ Inadequate awareness may lead to a depreciation in the acceptability of telemedicine. Massive campaigns including electronic, social and print media along with efficient infrastructure development are needed to overcome the deficiencies in the healthcare system towards health for all concepts due to the limited resources available.⁶⁵ The existing technological, regulatory and socio-economic barriers must be overcome by adopting a strategic and multidimensional approach incorporating digital interface, regulatory frameworks and mass awareness programmes.⁶⁶ The successful integration of telemedicine with existing systems requires a blend of efforts in all subdomains discussed in the study to achieve potential outcomes.

The strength of this study is that it is comprehensive and includes sample presentations of trained healthcare practitioners from all tiers of the healthcare system serving in telemedicine clinics. The study represents participation from most specialties rather than a specific specialty. The study is based on the responses of experienced practitioners who were performing telemedicine as per the exclusion and inclusion criteria. This measure has minimized the bias related to understanding the services provided by nonperformers and has helped us to bring out evidence based on first-hand experience and reviews of real-time performers. The pillar of the research was the analysis of healthcare professionals’ perspective on various key performance indicators of telemedicine service, such as patient comfort, privacy, safety and satisfaction; technology assessment; compatibility with the healthcare system; perceived impact; capacity building; and communication gap assessment. One of the limitations of our study is narrow representation of some specialties, as these specialties were underutilized at telemedicine clinics due to the unavailability of trained staff or the lower representation of reported patients. The limited access to the internet and willingness of healthcare providers to participate in surveys may have led to underrepresentation or overrepresentation of responses. This limitation is overcome by taking a larger sample that includes all the valid survey responses received. Another limitation which may be observed is that all questionnaire inputs came from HCPs, while MAST reports information from everyone involved in the process. The said study is particularly focusing on HCP perception to be added as an innovative component in the MAST model which is previously not incorporated in it.

Prospective cohort or longitudinal studies for measuring clinical efficacy of telemedicine may be carried out by bundling domains of MAST model like clinical effectiveness, patient perspective and organizational aspects²³ with HCP perception. The study will assist in developing models and frameworks for effective and efficient outcome-based implementation of telemedicine services encompassing interactions and interdependencies among multiple domains. It is suggested that additional research is needed in the field of telemedicine to develop long-term evidence-based and data-driven approaches that will allow operational integration of telemedicine with the current system. The incorporation of the HCP perception will help the policy makers in Pakistan to make an effective legislature aligned with the local challenges, literacy rate and technology available. This shall also help in devising a scorecard for benchmark and continuous performance monitoring toolkit to evaluate telemedicine healthcare services.

Conclusion

Telemedicine has a huge potential to be integrated with existing healthcare systems, and its compatibility along with clinical impact on healthcare facilities may be further expanded by effective measures for patient facilitation, technology ease, capacity building, regulations, ethical integrity and communication barriers. The MAST framework can be strengthened with the inclusion of HCP perception, as service providers are more familiar with the system. Since service providers are the intermediaries between the user and the services, their knowledge of the system makes them valuable additions to the MAST architecture. By expanding systematic integration in healthcare facilities with coordinated efforts toward all important variables of subdomains of HCP perception, it is possible to enhance the overall impact of telemedicine and its compatibility with the healthcare system. The various models affecting attitude and behaviours may also be studied in conjunction with MAST model for better understanding of telemedicine adoptability. Healthcare professionals’ (HCP) perceptions can significantly inform policymakers in Pakistan for the effective implementation and continuous improvement of telemedicine services by identifying gaps in key areas such as technological barriers, user interface and experience (UI/UX), shaping training programmes, enhancing patient care, remote monitoring and evaluation, efficient resource allocation, adjusting policies to cater cultural relevance, building trust and engagement, facilitating collaboration and addressing ethical concerns.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241297317 - Supplemental material for Aligning practitioner's perception: Empowering MAST framework for evaluating telemedicine services

Supplemental material, sj-docx-1-dhj-10.1177_20552076241297317 for Aligning practitioner's perception: Empowering MAST framework for evaluating telemedicine services by Ayesha Parvez, Javeria Saleem, Muhammad Ajmal Bhatti, Arshad Hasan, Asif Mahmood, Zulfiqar Ali and Tauseef Tauqeer in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241297317 - Supplemental material for Aligning practitioner's perception: Empowering MAST framework for evaluating telemedicine services

Supplemental material, sj-docx-2-dhj-10.1177_20552076241297317 for Aligning practitioner's perception: Empowering MAST framework for evaluating telemedicine services by Ayesha Parvez, Javeria Saleem, Muhammad Ajmal Bhatti, Arshad Hasan, Asif Mahmood, Zulfiqar Ali and Tauseef Tauqeer in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors are grateful to the administrative staff of Primary and Secondary Healthcare (PSHC), King Edward Medical University (KEMU) and University of the Punjab for providing the necessary support for this study. This work was supported by the Higher Education Commission of Pakistan under the Grant titled “Establishment of National Center of Robotics and Automation” (DF-1009-31).

Contributorship

AP: lead author; designing research, curation, conceptualization; methodology, formulating research, collecting, analyzing critical data. JS: designing research methodology, analyzing results, and shaping the intellectual content of the paper. MAB: data provision in the required format and management for the overall research framework. AH: statistical analysis of the data and contributed in the manuscript. AM: variables mapping, data analytics and support in interpreting outcomes. ZA: statistical analysis of the data, validation and ensuring scientific accuracy. TT: technical support, online surveys, manuscript preparation and data analysis. All authors have approved the final version of the manuscript to be published.

Data availability

Data are available upon reasonable request from the corresponding author.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Ethical Institutional Review Board at Information Technology University (ITU) (Ref: ITU/EIRB/2020/02), granted approval for this study to be conducted. All subjects provided consent to participate in the study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Researcher's Supporting Project Number (RSP2024R43), King Saud University, Riyadh, Saudi Arabia.

ORCID iD

Ayesha Parvez

Supplemental material

Supplemental material for this article is available online.

References

Freiburger

Holcomb

Piper

. The STARPAHC collection: part of an archive of the history of telemedicine. J Telemed Telecare 2007; 13: 221–223.

WHO. mHealth: new horizons for health through mobile technologies: second global survey on eHealth. 2011; viii, p.102.

Edworthy

. Telemedicine in developing countries. Br Med J 2001; 323: 524–525.

Yuen

Chua

, et al. The determinants of users’ intention to adopt telehealth: health belief, perceived value and self-determination perspectives. J Retail Consum Serv 2023; 73: 103346.

Carenzo

Costantini

Greco

, et al. Hospital surge capacity in a tertiary emergency referral centre during the COVID-19 outbreak in Italy. Anaesthesia 2020; 75: 928–934.

Lorenz

Lantzsch

. Theories predicting End-user acceptance of telemedicine use: systematic review. J Med Internet Res 2019; 21: e13117.

Yuen

Chua

, et al. The determinants of users’ intention to adopt telehealth: health belief, perceived value and self-determination perspectives. J Retail Consum Serv 2023; 73: 103346.

Abdool

Abdallah

Akhlaq

, et al. User acceptance level of and attitudes towards telemedicine in the United Arab Emirates. Sultan Qaboos Univ Med J 2021; 21: e203–e209.

Wang

Cao

. Factors influencing continuous intention to use telemedicine after the COVID-19 pandemic in China: an extended technology acceptance model. Open J Soc Sci 2022; 10: 344–359.

10.

Assegaff

Kurniabudi

Sharipuddin

. A literature review: acceptance models for e-learning implementation in higher institution. 2014.

11.

Kidholm

Clemensen

Caffery

, et al. The model for assessment of telemedicine (MAST): a scoping review of empirical studies. J Telemed Telecare 2017; 23: 803–813.

12.

Mehraeen

SeyedAlinaghi

Heydari

, et al. Telemedicine technologies and applications in the era of COVID-19 pandemic: a systematic review. Health Informatics J 2023; 29: 14604582231167432.

13.

Siripipatthanakul

Limna

Sriboonruang

, et al. Applying the TPB and the UTAUT models predicting intentions to use telemedicine among Thai people during the COVID-19 pandemic. Int J Comput Sci Res 2023; 7: 1362–1384.

14.

Nunnery

Klocko

, et al. Lung cancer survivors’ views on telerehabilitation following curative intent therapy: a formative qualitative study. BMJ Open 2023; 13.

15.

Baker

. The technology–organization–environment framework. In: Yogesh K.

Wade

(eds) Information systems theory: explaining and predicting our digital society, vol. 1. New York, NY: Springer New York, 2012, pp.231–245.

16.

Hajesmaeel-Gohari

Bahaadinbeigy

. The most used questionnaires for evaluating telemedicine services. BMC Med Inform Decis Mak 2021; 21: 36.

17.

Parmanto

Lewis, Jr.

Graham

, et al. Development of the telehealth usability questionnaire (TUQ). Int J Telerehabil 2016; 8: 3–10.

18.

Russell

Champion

Skinner

. Psychosocial factors related to repeat mammography screening over 5 years in African American women. Cancer Nurs 2006; 29.

19.

Orji

Vassileva

Mandryk

. Towards an effective health interventions design: an extension of the health belief model. 2012.

20.

Jones

Jensen

Scherr

, et al. The health belief model as an explanatory framework in communication research: exploring parallel, serial, and moderated mediation. Health Commun 2015; 30: 566–576.

21.

Williams

Cervone

. Social cognitive theories of personality. In: Advanced personality. New York, NY: Plenum Press, 1998, pp.173–207.

22.

Saigí-Rubió

Torrent-Sellens

Ramos

, et al. Framework for the implementation of a telemedicine service. Washington, DC: PAHO, 2016.

23.

Kidholm

Clemensen

Caffery

, et al. The model for assessment of telemedicine (MAST): a scoping review of empirical studies. J Telemed Telecare 2017; 23: 803–813.

24.

Kidholm

Ekeland

Jensen

, et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care 2012; 28: 44–51.

25.

Ekeland

Grøttland

. Assessment of MAST in European patient-centered telemedicine pilots. Int J Technol Assess Health Care 2015; 31: 304–311.

26.

Bornelöv

Komorowski

. Selection of significant features using monte carlo feature selection. Stud Comput Intell 2016; 605: 25–38.

27.

Draminski

Koronacki

. Rmcfs: An R package for Monte Carlo feature selection and interdependency discovery. J Stat Softw 2018; 85.

28.

Shang

Meng

Wang

, et al. Local discriminative based sparse subspace learning for feature selection. Pattern Recognit 2019; 92: 219–230.

29.

Huang

J-C

. Innovative health care delivery system – A questionnaire survey to evaluate the influence of behavioral factors on individuals’ acceptance of telecare. Comput Biol Med 2013; 43: 281–286.

30.

Demiris

. Examining health care providers’ participation in telemedicine system design and implementation.

31.

Gagnon

Orruño

Asua

, et al. Using a modified technology acceptance model to evaluate healthcare Professionals’ adoption of a new telemonitoring system. Telemed e-Health 2012; 18: 54–59.

32.

Park

H-Y

Kwon

Y-M

Jun

H-R

, et al. Satisfaction survey of patients and medical staff for telephone-based telemedicine during hospital closing due to COVID-19 transmission. Telemed e-Health 2021; 27: 724–732.

33.

Kasim

Salih

Attash

. Usability of telehealth among healthcare providers during COVID-19 pandemic in nineveh governorate, Iraq. Public Health Pract 2023; 5: 100368.

34.

Indria

Alajlani

Fraser

HSF

. Clinicians perceptions of a telemedicine system: a mixed method study of makassar city, Indonesia. BMC Med Inform Decis Mak 2020; 20: 233.

35.

Albarrak

Mohammed

Almarshoud

, et al. Assessment of physician’s knowledge, perception and willingness of telemedicine in Riyadh region, Saudi Arabia. J Infect Public Health 2021; 14: 97–102.

36.

Acharya

Rai

. Evaluation of patient and doctor perception toward the use of telemedicine in Apollo Tele Health Services, India. J Family Med Prim Care 2016; 5.

37.

Ayatollahi

Sarabi

FZP

Langarizadeh

. Clinicians’ knowledge and perception of telemedicine technology. Perspect Health Inf Manag 2015; 12: 1c.

38.

Cochran

. Sampling techniques. 3rd ed. New York: John Wiley & Sons, 1977.

39.

Gupta

Joshi

, et al. Is telemedicine a holy grail in healthcare policy: clinicians’ and patients’ perspectives from an Apex Institution in Western India. BMC Health Serv Res 2023; 23.

40.

Asua

Orruño

Reviriego

, et al. Healthcare professional acceptance of telemonitoring for chronic care patients in primary care. BMC Med Inform Decis Mak 2012; 12: 139.

41.

Tan

AJQ

Rusli

KDB

McKenna

, et al. Telemedicine experiences and perspectives of healthcare providers in long-term care: a scoping review. J Telemed Telecare 2024; 30: 230–249.

42.

Dharmar

Kuppermann

Romano

, et al. Telemedicine consultations and medication errors in rural emergency departments. Pediatrics 2013; 132: 1090–1097.

43.

Glock

Nymberg

Bolmsjö

, et al. Attitudes, barriers, and concerns regarding telemedicine among Swedish primary care physicians: a qualitative study. Int J Gen Med 2021; 14: 9237–9246.

44.

Guise

Anderson

Wiig

. Patient safety risks associated with telecare: a systematic review and narrative synthesis of the literature. BMC Health Serv Res 2014; 14: 588.

45.

De Micco

Fineschi

Banfi

, et al.

From COVID-19 Pandemic to Patient Safety: A New “Spring” for Telemedicine or a Boomerang Effect?

Front Med 2022; 9.

46.

Asrat

Tarekegn

Abderehim Yesuf StPeter

, et al. Diabetic patients’ satisfaction of teleconsultations and associated factors during COVID-19 pandemic at Tikur Anbessa Specialized Hospital, Addis Ababa, Ethiopia Leilina Shimeles Talent Youth Association. Published online first 2023. doi: 10.21203/rs.3.rs-3225793/v1.

47.

Sun

Tan

, et al. Usage and perceptions of telemedicine among health care professionals in China. Int J Med Inform 2022; 166: 104856.

48.

Choi

Kim

Park

, et al. Telemedicine in otolaryngology during COVID-19: patient and physician satisfaction. Otolaryngol Head Neck Surg 2022; 167: 56–64.

49.

SteelFisher

McMurtry

Caporello

, et al. Video telemedicine experiences in COVID-19 were positive, but physicians and patients prefer in-person care for the future. Health Aff 2023; 42: 575–584.

50.

Pinedo-Torres

Garcia-Villasante

Gutierrez-Ortiz

, et al. The doctor-patient relationship and barriers in non-verbal communication during teleconsultation in the era of COVID-19: a scoping review. F1000Res 2023; 12: 676.

51.

Rahman

Amit

Kafy

A-A

. Gender disparity in telehealth usage in Bangladesh during COVID-19. SSM Ment Health 2022; 2: 100054.

52.

Barreiro

Coles

Hales

, et al. Barriers to the implementation of telehealth in rural barriers to the implementation of telehealth in rural communities and potential solutions communities and potential solutions.

53.

Mahdi

Allana

Battineni

, et al. The promise of telemedicine in Pakistan: a systematic review. Health Sci Rep 2022; 5.

54.

Alkureishi

Choo

Z-Y

Lenti

, et al. Clinician perspectives on telemedicine: observational cross-sectional study. JMIR Hum Factors 2021; 8: e29690.

55.

Ariens

LFM

Schussler-Raymakers

FML

Frima

, et al. Barriers and facilitators to eHealth use in daily practice: perspectives of patients and professionals in dermatology. J Med Internet Res 2017; 19: e300.

56.

Shawwa

. The use of telemedicine in medical education and patient care. Cureus. Published online first: 18 April 2023. doi: 10.7759/cureus.37766.

57.

Bajra

Srinivasan

Torres

, et al. Training future clinicians in telehealth competencies: outcomes of a telehealth curriculum and teleOSCEs at an academic medical center. Front Med 2023; 10.

58.

Merrill

Roe

Seely

, et al. Advanced telemedicine training and clinical outcomes in type II diabetes: a pilot study. Telemed Rep 2022; 3: 15–23.

59.

Stovel

Gabarin

Cavalcanti

, et al. Curricular needs for training telemedicine physicians: a scoping review. Med Teach 2020; 42: 1–9.

60.

Kazmi

Yasmin

Siddiqui

, et al. Nationwide Assessment of Knowledge and Perception in Reinforcing Telemedicine in the Age of COVID-19 Among Medical Students From Pakistan. Front Public Health 2022; 10.

61.

Zhang

Shi

. The influence of telemedicine on capacity development in public primary hospitals in China: a scoping review. Clin eHealth 2022; 5: 91–99.

62.

Asua

Orruño

Reviriego

, et al. Healthcare professional acceptance of telemonitoring for chronic care patients in primary care. BMC Med Inform Decis Mak 2012; 12: 139.

63.

Sarah

Patel

, et al. Understanding Physicians’ preferences for telemedicine during the COVID-19 pandemic: cross-sectional study. JMIR Form Res 2021; 5: e26565.

64.

Mitra

Veerakone

, et al. Telemedicine in paediatric emergency care: a systematic review. J Telemed Telecare 2023; 29: 579–590.

65.

Bokolo

Chi Phan

Anthony McGrowder

, et al. Impact of the COVID-19 pandemic on knowledge, perceptions, and effects of telemedicine among the general population of Pakistan: A national survey.

66.

Irfan

Yaqoob

. The digital lifeline: telemedicine and artificial intelligence synergy as a catalyst for healthcare equity in Pakistan. Cureus 2024; 16: e54017.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.21 MB