Abstract
Objective
With the digitalization of objects and spaces, healthcare services are being reshaped globally, creating many potential applications. This study aimed to determine the application potential of remote healthcare services (RHS) in a hospital by considering the experiences, interests, and suggestions of health professionals, and examples of useful applications that can be used, developed, or invented for healthcare systems.
Methods
A semi-structured, face-to-face interview survey was conducted with 176 healthcare professionals working at Bozok University.
Results
Branches with the highest practice experience were internal medicine, cardiac, pediatric, infection, and orthopedics. Experienced participants rated the usability of “Consultation,” “Support,” and “Monitoring” applications higher than other apps, and indicated that they would prefer to use them for themselves (η²≤0.12). Requirements adequacy was lower for older adults, internal/surgical branches, and physicians than for other groups (η² ≥ 0.05). Application categories showed a significant relationship (0.4 ≤ r ≤ 0.8, p < 0.05). Several variables significantly explained the models (p < 0.001): application-usability (64%), user-demand (64%), and requirements-adequacy (25%). Professionals’ demand (r = 0.83) was more strongly correlated to usability than patient demand (r = 0.63). Health professionals (N = 105) from 17 branches provided 57 available, 51 developable, and 19 innovative application recommendations. These were coded according to application type, critical features, presence, integration status, and usefulness.
Conclusion
RHS’ application potential in a hospital was revealed considering demographic factors and application categories based on health professionals’ experiences, practical interests, and suggestions, with a strong, comprehensive, and up-to-date methodology. The findings have the potential for international application and can contribute to implementing useful and developing original applications.
Keywords
Background
Currently, smart technologies are developing at an unprecedented pace. Smart environments and objects are being designed, and complex interventions in biological structures are being performed remotely by leveraging the sensitive sensors of smart devices. 1 The use of new technologies in healthcare and medicine has created tremendous application potential, especially given the digital transformations necessitated by the COVID-19 pandemic.2,3 For instance, applications that make impossible procedures possible, such as remote robotic surgeries, have become widespread. Moreover, important factors that advantage users are finding solutions to the obstacles arising from spatial, temporal, and individual limits, facilitating, accelerating, and simplifying the work procedures by transferring some of them to technology, 4 thus preventing human-caused errors. 5 Therefore, the concept of digital healthcare plays an important role in providing more up-to-date, efficient, and quality healthcare services while benefiting individuals. 6
As digital health reforms progress globally, the situation in hospitals—the cornerstones of the healthcare system—is becoming particularly intriguing. In the developing version of remote healthcare services (RHS), with the transformative effect of new health technologies, hospitals are moving from being places of observation and intervention to remote control centers. In this framework, institutions should not only overcome their deficiencies but also promptly take advantage of innovation opportunities. Hospitals should structure their information systems, and establish their application laboratories and markets.
In its traditional sense, RHS is generally used by private healthcare providers, and is a format that includes only patient and physician meetings. There is also conceptual confusion among researchers and users about whether the terms tele, e, mobile, online, virtual, smart, web-based and digital health are within the scope of the RHS. Therefore, traditional definitions must be updated within the framework of technological development before designing a study on the subject. 7
However, the transition to new practices depends on established orders and standards. 8 For applications that are in use and can be improved, studies have evaluated certain types of RHS.9,10 Furthermore, the applicability of RHS has been investigated using a mixed methodology specific to certain health branches and generally for commonly used applications. 11 Additionally, commonly used applications that healthcare professionals recommend to their patients have been addressed in structured quantitative studies. However, these designs provide a limited understanding of the basic and indispensable issues in developing or adapting Internet-based RHS interventions. 12
Studies have also highlighted the potential for digital health security and artificial intelligence (AI)-driven applications. For example, AI-driven applications in preventive health can be considered an alternative treatment approach for infectious diseases. 13 Alternately, secure, interactive multimedia applications by enhancing remote media areas with heterogeneous image sources; can also be used to provide healthcare support in non-hospital settings. 14
The majority of future innovation recommendations are based on literature reviews or qualitative studies with small groups of participants.7,15,16 Furthermore, e-health applications are often designed by researchers without consulting the primary users. These studies do not consider the needs of end users but require intense effort, resulting in poor usability, frustration, and limited acceptance.12,17 Therefore, more studies are required on both the multifaceted evaluation of the usability of applications and developing new applications.18–22 Physicians’ adoption of telemedicine can be explained by behavioral usage intention, actual usage, satisfaction, attitude, continuous usage and recommendation intention variables. 23 Also a deeper exploration of user needs can provide smarter healthcare solutions that can better meet practical requirements. 24
Addressing these gaps, this study evaluates the application potential of RHS in a hospital, considering application type, healthcare branch, and professionals. It aims to answer the question, “İf experiences, practical interests, and recommendations of health professionals improve RHS applicability, what applications can be used, developed, or invented considering these factors?” In particular, the following specific questions are answered:
Qualitative questions: 1. What are the types of RHS and applicability categories? 2. Is there another app professionals would like to use in their work? 3. What applications are health professionals interested in or would like to see developed?
Quantitative questions: 1. In which branches and to what extent are the RHS types used? 2. Is there a relationship between applicability categories and the usability of RHS types? 3. Does RHS applicability differ among different demographic groups? 4. How do usefulness, adequacy, experience, demand, education, and age play a role in RHS applicability categories and app-type usability?
Common inference questions: (1) What are the coded recommendations based on application type and other emerging themes? (2) What are the common inferences from the quantitative and qualitative questions? (3) What are the RHS application potentials for the hospital regarding existing, developable, or innovative applications?
Overall, this study has important strengths as the use of both qualitative and quantitative methods provides a well-rounded understanding of the topic. For new research topics, exploratory and explanatory data are required. While qualitative data can provide sufficient insight, quantitative data can provide easy, effective, and powerful measurements. Description and generalization become easier as the scope and diversity of participants increase.
Moreover, this study addresses practical concerns and suggestions from healthcare professionals, making the findings highly relevant for improving RHS. Additionally, including healthcare professionals from various branches ensures a comprehensive view of the current state and needs of RHS.
Methods
Research design
This study employed a cross-sectional, descriptive, and analytical design. The big picture of the study problem was explained using quantitative data, important codes and themes were explored using qualitative data. In addition the quantitative data were complemented and explained with qualitative data. A complex mixed-methods design was used instead of a single method design. 25 Specifically, the study was conducted in three stages: an exploratory sequential design, followed by an explanatory sequential design, and finally, a concurrent design. 26 This study utilized the six basic criteria determined by Hirose and Creswell for mixed-method studies. 25 Scheme 1 presents the research design procedure and describes the data collection, analysis methods, and associated outputs. The research design and reporting were in accordance with the standards specified in “Mixed Methods Reporting in Rehabilitation & Health Sciences.” 27
Researcher Background and Contributions to the Project
Environment and scope
The research was conducted in a university hospital in Turkey between September 25, 2022, and March 25, 2023. Turkey's health institutions have their own information systems. Additionally, Turkey's personal health record application, “E-Nabız,” is used under the control of the Ministry of Health.
Generally, health professionals with a medical background work in at least one different health institution before being appointed to an academic position at a university hospital. Other employees may transfer from private hospitals or different health institutions. Transfers from private hospitals are common in the hospital where the study was conducted.
Permission and ethical approval to conduct the study was received from the Hospital Management and University Ethics Commission (September 21, 2022, No.: 36/18). The purpose of the research and confidentiality of the data were explained to the participants, and informed verbal and written consent was obtained.
Participants
The study population comprised 301 healthcare professionals actively working in 21 branches of a university hospital clinic. To reach a statistically significant sample size, the sample size was calculated as 170 people with a confidence interval of 95% and a margin of error of 5%. Interviews were conducted with 191 people, comprising managers, assistant managers, supervisors, and unit employees, who agreed to participate in the interview voluntarily with the guidance of the unit manager. Subsequently, 15 participants who left 5% or more of the survey items blank, double-ticked, or had inconsistencies in their answers were excluded. Furthermore, 105 participants from 17 branches made RHS application recommendations. At least one person from each branch participated in the study. In qualitative research, a large amount of information can be obtained with a small sample size in studies conducted with participants who are experts with high knowledge power. 28 Participants presented examples of RHS relevant to their field and examples of other field applications that make their field possible. This is significant when considering the interdependence among branches in providing health services.
Healthcare professionals have a very heterogeneous specialization structure. This study ensured participation from every active branch in the hospital, especially by unit heads or their assistants. Furthermore, non-physician healthcare professionals can work in mixed units comprising more than one branch. Therefore, participants who did not work in a specific branch were included in the mixed branch.29,30 Subgroup comparisons were made if the number of participants was at least 10 or if statistical significance was achieved according to the analysis. Ultimately, data obtained from 176 participants grouped into 15 branches were analyzed (Figure 1).
Participant flowchart.
Data collection
Pilot study
In the first phase of this study, aimed at determining the RHS types and applicability categories, trial interviews were conducted with participants representing internal, surgical and mixed specialties. Based on this, eight themes of RHS categories were determined, two themes of “user demand,” and three themes of “adequacy of requirements.” Subsequently, adjustments were made to the interview protocol based on expert experience and information provided by the participants. The semistructured interview form was revised, unnecessary expressions were removed, and new ones were added or changed 31 (Appendix 1). Detailed information is given in Scheme 1.
Instrumentation
Remote Health Services and Application Categories
Many current definitions of health have emerged with the combination of technological and health terms, such as tele, electronic (e), mobile, online, virtual, smart, web-based, and digital health/medicine. These refer to a broad or narrow scope of services depending on the meaning of the added word and technological features. For example, e-health is the new name for health in the technology age and is defined as the provision of health services electronically in a web-based environment. 32 New health technologies are changing and improving how health services and many traditional practices are implemented. For instance, AI, augmented reality, sensor sensing, blockchain, Internet-of-Things (IoT), metaverse, and other technologies are making it more realistic to consider health in new contexts and have also developed tools to increase physical independence.16,33
The traditional name for RHS is telehealth. Telehealth, in addition to tele, includes medicine, consultancy, monitoring, and several other aspects. Here, diverse definitions are created by adding words. The most important differentiator in telehealth is physical distance. Many applications, such as monitoring with connectivity technologies, robotic surgery, drug treatment, implants, diagnosis, and care, can be performed over long distances. 34 Therefore, RHS can be defined as remote delivery of healthcare services through digital technologies such as wearable smart devices, application software, information communication, and connectivity technologies. Besides distance, RHS covers other new definitions and key areas of healthcare such as electronic or operational.
The application categories were determined according to the general service areas of health services provided in the hospital based on the literature and extant practices.32–34 They were revised in the first and second phases, and at the end of the study.
Applicability of Remote Health Services
In prior studies, to evaluate the usability of an application, participants who had experienced an application were usually asked about statements that measure the usefulness, practicality, reliability, user satisfaction, and other aspects of the application. 11 However, three main themes have been highlighted in the research implications: “usability”1,11,35–37; “adequacy” 11 —technical adequacy,1,36–40 worker count adequacy, 38 worker qualification adequacy,28,35,40–42 reliability,20,43 and financial and legal adequacy; and “demand”20,44,45—profession, patient, 35 supplier, and politician demand. The current study investigated the usability of applications that employees not only experience but are also interested in. Therefore, the usability of each RHS type with specific characteristics was investigated. To determine the application experience, participants were asked to indicate their previous use of the application types via yes or no options. During the trial phase, the reliability, financial adequacy, legal adequacy, and supplier demand categories were eliminated. Healthcare professionals perceive these issues as technical aspects under the control of institutional owners, experts, or management. The applicability categories included in the research questionnaire are shown in Figure 2.
Applicability categories.
Implementation
The interview method was used to collect data. During the interview, participants were asked to evaluate the RHS categories specifically for their branches. The responses were transcribed verbatim using a voice recording or note-taking application with the participant's permission. The content was reviewed, coded, and anonymized.
Data analysis
Principal component analysis (PCA) was conducted to determine constructs in the applicability questionnaire: “app usefulness,” “user demand,” and “adequacy of requirements.” IBM SPSS Program Statistics 29 (IMB Corp., NY, USA) was used for the quantitative analysis. Compliance with a normal distribution was examined using the Kolmogorov–Smirnov test. Additionally, skewness and kurtosis values were examined. If these values are between ±2, continuous scores show normal distribution. 46 Descriptive statistics are provided for the sociodemographic and applicability categories. A one-way analysis of variance (ANOVA) was used to compare independent groups. Significant results in the ANOVA test were examined using the multiple comparison tests to determine which group the difference was in. Pearson's correlation (r) was used to measure the linear relationship between the applicability categories. Subsequently, multivariate linear regression was performed to measure the role of usefulness, adequacy, experience, demand, education, and age in the applicability categories of RHS and app-type usability. The basic assumptions required for statistical comparison methods were tested. The statistical significance level was set at 5%. Content and thematic analyses were applied to the sample application ideas in MAXQDA 2022. The data were presented with weighted code models and crosstabs using the frequency distribution of coded documents or sections. Key findings from the qualitative and quantitative analyses were combined into one table to illustrate the common conclusions. Thus, stronger findings can be achieved by adding the insights developed from integrating qualitative and quantitative data. 25
Reliability and validity tests
Cronbach's alpha and PCA were calculated for each construct in the questionnaire.
Usefulness: Cronbach's alpha: 0.925; PCA: eight components (“physical examination,” “measurement,” “evaluation,” “treatment/care,” “operation,” “monitoring,” and “counseling”) explaining 65.9% of the variance.
User Demand: Cronbach's alpha: 0.822; PCA: two components (“own-demand” and “patient demand”) explaining 84.9% of the variance.
Adequacy of requirements: Cronbach's alpha: 0.924; PCA: three components (“technical adequacy,” “worker count adequacy,” and “worker qualification adequacy”) explaining 91.8% of the variance.
To ensure the reliability of the qualitative analysis, data, analyst, and method triangulation was applied, and application categories determined from the literature were used. 47
Results
Evaluation of the applicability of remote health applications
Participant profile
A total of 53% (94 people) and 47% (81 people) of the participants were in the 20–30 years and 30–67 age groups, respectively. Furthermore, 39% (69 people) had an undergraduate education, while 42.6% (75 people) had a postgraduate education. Finally, 48.3% (85 people) were nurses, 40.3% were physicians (71 people), and 11.4% (20 people) were other health workers (Table 1). Regarding working experience in professional branches, 95 (54%) participants had been working for 5 years or less, 61 (34.7%) for 5–15 years, and 20 (11.4%) for more than 15 years. The distribution of participants by branch was 10 at the lowest and 15 at the highest.
Participant Profile
Note: Mixed (…S) Both internal and surgical branches.
Determining sample application usage experience according to branch
A total of 175 (99.4%) participants reported having used RHS applications before. Of these, 110 (62.5%) had used any RHS application category through an informal or official digital application. The branches with the highest practice experience were internal medicine (13 people, 92.9%), cardiac (12 people, 80%), and pediatrics (8 people, 80%). Participants mostly used consultation (cardiac: 11 people; surgery/child/infection: 9 people; total: 89 people, 51.4%), support (internal medicine: 9 people; orthopedics: 5 people; total: 37 people, 21.6%), and monitoring applications (internal medicine: 8 people; pediatrics: 6 people; orthopedics: 5 people; total: 33 people, 19.2%) (Table 2).
Sample application usage experience according to branch.
Note: Mixed (…S) Both internal and surgical branches.
Pearson correlation analysis: identifying relationships affecting applicability
Pearson's correlation analysis was used to determine the relationships between “adequacy” and “demand” with the usability of examination, measurement, evaluation, treatment/care, operation, monitoring, counseling, and support applications (Table 3).
Relationships between usability of application types, adequacy of requirements and user demand.
p < 0.01. Significance levels are two-way.
“Care usability” and “own-demand” exhibited a highly positive (r = 0.731) and significant (p < 0.05) relationship, with 53.4% of the variation within the “care usability” variable being explained by the “own-demand” variable. Participants reported wanting to use care applications that they found useful in their work. In general, all variables exhibited a moderate (r = 0.40–0.59) to strong (r = 0.60–0.79) significant (p < 0.05) relationship. The relationships between the “adequacy of requirements” variable, and measurement, operation, examination, consultation, support, and “own-demand” variables were weak (r = 0.20–0.39) but significant (p < 0.05).
One-way ANOVA: identifying demographic group differences regarding RHS applicability
A one-way ANOVA was conducted to determine demographic group differences regarding RHS applicability. Significant differences were found between the groups, especially regarding the adequacy of requirements (Table 4).
Identifying Demographic Group Differences Regarding RHS Applicability
* P < 0,05, η²; Estimated eta-squared.
The mean scores related to the adequacy of requirements showed a significant difference by age group (F[4–170] = 3.33, p < 0.05). The multiple comparison test revealed that the perceptions of the adequacy of requirements among those in the under-25-years age group (M = 2.64, SD = 0.513) were significantly (η²=0.07) higher than among those in the 36–40-years (M = 1.88, SD = 0.786) and 41+-years age groups (M = 1.96, SD = 0.820). The results were similar for all three subcategories of the adequacy of requirements.
Furthermore, comparing the applicability means of the education groups revealed differences between the groups in “count adequacy “ (F[4–171] = 2.509, p < 0.05) and “care usability” (F[4–171] = 2.781, p < 0.05). The perceptions of the “doctorate” group regarding “count adequacy” (M = 1.88, SD = 0.881) were moderately (η²=0.055) lower than those of the “high school” (M = 2.41, SD = 0.913) and “license” groups (M = 2.44, SD = 0.839). Furthermore, the perceptions of the “master” group regarding “care usability” (M = 3.32, SD = 1.192) were significantly (η²=0.061) higher than those of the “high school” (M = 2.65, SD = 1.018) and “license” groups (M = 2.75, SD = 1.108).
Next, comparing the applicability means of the profession groups revealed significant differences in “adequacy” (F[2–173] = 6.053, p < 0.05) and “support usability” (F[2–173] = 4.162, p < 0.05). The perceptions of the “physician” group regarding “adequacy” (M = 1.98, SD = 0.808) were significantly (η²=0.065) lower than those of the “other” (M = 2.42, SD = 0.920) and “nurse” groups (M = 2.41, SD = 0.784). The results for the “adequacy” subcategories were similar. However, the perceptions of the “other” group regarding “support usability” (M = 2.76, SD = 1.108) were moderately (η²=0.046) lower than those of the “physician” (M = 3.53, SD = 1.164) and “nurse” groups (M = 3.40, SD = 0.970).
Subsequently, comparing applicability means of experience groups revealed significant differences in “own-demand” (F[2–173] = 3.533, p < 0.05), “monitoring usability” (F[2–173] = 4.186, p < 0.05), “consultation usability” (F[2–173] = 11.412, p < 0.05), and “support usability” (F[2–173] = 5.508, p < 0.05). The perceptions of the “two app+” group regarding “own-demand” (M = 3.35, SD = 0.834) were moderately (η²=0.039) higher than those of the “none” (M = 2.92, SD = 0.959) and “one app” groups (M = 3.03, SD = 0.964). Moreover, the perception of “consultation usability” of the “none” group (M = 2.93, SD = 1.187) was significantly (η²=0.117) lower than those of the “two app+” (M = 3.78, SD = 0.850) and “one app” groups (M = 3.55, SD = 0.997). Furthermore, the perception of “support usability” of the “two app+” group (M = 3.75, SD = 0.899) was significantly (η²=0.060) higher than those of the “none” (M = 3.13, SD = 1.081) and “one app” groups (M = 3.28, SD = 1.194). similarly, the “two app+” group's perceptions of “monitoring usability” (M = 3.47, SD = 0.821) were moderately (η²=0.046) higher than those of the “none” (M = 2.96, SD = 1.083) and “one app” groups (M = 3.08, SD = 1.156).
Finally, comparing the applicability means of the branch groups revealed a significant difference in “adequacy” (F[2–173] = 4.512, p < 0.05). the perceptions of the “mixed” group regarding “Adequacy” (M = 2.68, SD = 0.930) were moderately (η² = 0.050) higher than those of the “surgical” (M = 2.13, SD = 0.766) and “internal” groups (M = 2.21, SD = 0.840). the results for the “adequacy” subcategories were similar.
Multiple linear regression analysis: the role of usefulness, adequacy, experience, demand, education, and age on RHS applicability categories and app type usability
The role of the applicability variables was evaluated together with the impact of demographic changes. For each applicability category, a model was created that included age, education, and experience variables along with other applicability categories. The most appropriate model was determined by removing variables that did not significantly contribute to the model. All applicability variables together significantly affected the model variables. Among the demographic variables, only education contributed significantly to the adequacy model (Table 5).
The Role of Usefulness, Adequacy, Experience, Demand, Education, and Age on RHS Applicability Categories and App Type Usability
F: ANOVA F value showing the significance of the model; DW: Autocorrelation between variables Durbin Watson; VIF: Variance Inflation Factor;
Among the effects of the model variables on “usefulness,” the “demand” (β = 0.749, p < 0.001) and “adequacy” (β = 0.106, p < 0.05) variables significantly explained 63.8% of the model (R2 = 0.638, F = 152.478, p < 0.001). Next, regarding the effects on “demand,” the “usefulness” (β = 0.751, p < 0.001) and “adequacy” (β = 0.101, p < 0.05) variables significantly explained 63.7% of the model (R2 = 0.637, F = 151.814, p < 0.001). regarding the effects on “adequacy,” the “demand” (β = 0.244, p < 0.05), “usefulness” (β = 0.248, p < 0.05), and “education” (β = −0.236, p < 0.001) variables significantly explained 24.6% of the model (R2 = 0.246, F = 18.722, p < 0.001). While a one-unit increase in the “education” variable reduced the “adequacy” variable by 0.24 units, a one-unit increase in the “demand” variable increased the “adequacy” variable by 0.24 units. Likewise, a one-unit increase in the “usefulness” variable increased the “adequacy” variable by 0.25 units.
Finally, examining the effects of model variables on “app type demand,” “care usability” (β = 0.393, p < 0.001), “monitoring usability” (β = 0.233, p < 0.001), “consultation usability” (β = 0.157, p < 0.05), and “support usability” (β = 0.215, p < 0.001) variables significantly explained 70.2% of the model (R2 = 0.702, F = 98.821, p < 0.001). Furthermore, examining the effects on “app type adequacy,” “examination usability” (β = 0.289, p < 0.001) and “monitoring usability” (β = 0.247, p < 0.05) variables significantly explained 22.3% of the model (R2 = 0.223, F = 24.851, p < 0.001).
Example application ideas
The responses received from 105 participants in 17 branches were categorized by content analysis using MAXQDA 2022. Specifically, the application features, RHS type, integration into the RHS system, and applicability were examined. The number of coded sections (CS) was calculated.
Features specified in applications
The features emphasized as necessary or useful for the implementation of the application were coded. The number of coded sections was calculated. The main features required to use the application were identified as data storage and transmission (CS = 70), followed by recognition-detection (CS = 55), wearable portable device (CS = 55), multi-connection-integration (CS = 49), planning-setting (CS = 47), warning-command (CS = 42), notification (CS = 40), meeting (CS = 31), digital identity creation-verification (CS = 27), display (CS = 25), practicability (CS = 24), reporting (CS = 23), reminding (CS = 19), digital assistant (CS = 19), and location determination (CS = 18) (Figure 3). The application suggestions are given in Appendix 2.
Application feature; code sub-code model.
Application recommendations specific to health service type
Sample application ideas were coded according to RHS type. Although physical examination, evaluation, and monitoring are determined to be different service types in the literature, because there is no detailed content about these groups, the categories have been combined as diagnosis-monitoring types. Application recommendations can be categorized into many types of services by their content. In this study, coding was performed to cover all of application type; however, when selecting the examples given for the service type, the main emphasis in the application proposal was considered.
Diagnosis and monitoring applications
In this application group, scanning, monitoring, and evaluation recommendations were made for the branch, disease, and health status. The most frequently emphasized codes in this category were follow-up and monitoring (CS = 51), followed by physical examination (CS = 13), symptom scanning (CS = 10), emergency assessment (CS = 7), mobility assessment (CS = 7), and branch-specific evaluation 7 (Figure 4). Some application suggestions are given in Appendix 3.
Diagnosis and monitoring applications; code subcode model.
Biometric measurement applications
In these applications, vital values (CS = 27), blood sugar (CS = 11), blood pressure (CS = 11), imaging (CS = 11), mobility (CS = 6), and electrocardiography were the most frequently used codes (Figure 5). Some application suggestions are given in Appendix 4.
Biometric measurement applications; hierarchical code sub-code model.
Care and treatment applications
In these applications, the medication (CS = 12) and exercise (CS = 11) codes were most frequently emphasized (Figure 6). Thus, the recommended practices should ensure the effectiveness, efficiency, and controllability of treatment, especially for patients who are sent home. Some application suggestions are given in Appendix 5.
Care and treatment applications; code sub-code model.
Operation applications
The most frequently used codes in these applications were remote adjustment or use of devices/equipment by an expert user (CS = 12), emergency intervention (CS = 8), and drug administration (CS = 7) (Figure 7). Some application suggestions are given in Appendix 6.
Operation applications; code sub-code model.
Counseling and support applications
Among these applications, the most common code was counseling practice between the patient and healthcare provider (CS = 23), followed by suggestions regarding remote health environments (CS = 27) (Figure 8). Some application suggestions are given in Appendix 7.
Counseling and support applications; code sub-code model.
Integration status of the application into the hospital remote health system
Recommended applications were classified according to their development status, and their distribution according to branch was provided by creating the MAXQDA 2022 Cross Table (Table 6).
Distribution of application development status based on branch.
Coded sections were counted once per document.
The majority of recommendations were regarding the integration of existing applications into the system (N = 57, 44.88%). This was followed by recommendations for developing and integrating existing applications (N = 51, 40.16%). While internal branches mostly recommended integrating existing applications by improving them (internal: 24-surgical: 17), surgical branches mostly suggested integrating new applications (surgery: 13, internal: 4). Some application suggestions are given in Appendix 8.
Sample applications according to their application potential
The potential for using applications for each application recommendation was graded according to expression style, emotion analysis, and development status. The use of positive expressions about the application, such as beautiful, useful, good, and excellent, in the text content and application suggestion made by more than one participant were considered in the rating (Table 7).
Distribution of Application Potential and Integration Status Based on Main Brancha
The number of applications was determined by the sum of the same applications suggested by different participants or different applications suggested by the same participants.
The data in Table 7 were obtained using the analysis made with the MAXQDA 2022 Interactive Quote Matrix. Many RHS applications with high application potential exist for all branches. The majority of the recommendations related to integrating existing applications into the corporate system either directly or by improving them.
Qualitative-Quantitative integrated findings on RHS applicability
Different data types and analyses yielded findings that support, explain, or extend each other (Table 8).
Qualitative-quantitative integrated findings on RHS applicability.
* p < 0.05, **p < 0.01 level. η²: estimated eta-squared. Groups differences (GD: 1., 2., 3., 4. and 5.) were identified by “&;and, |;or” signs.
See Table 1 for more details. 2 See Figures for more details. 3See Appendices for more details.4See Table 3 for more details.5See Table 4 for more details.6See Table 5 for more details.7See Table 4 for more details.
“Experience” (N = 110) and “quan-usefulness” (M = 3.10) categories had high values in the same application types: “counseling” (N = 89/M = 3.40), “support” (N = 37/M = 3.38), and “monitoring” (N = 33/ M = 3.17). Applications with which users are more experienced are perceived as more usable. However, the applications that received high values in the “qual-usefulness” category were “support” (CS = 137), “evaluation” (CS = 105), and “measurement” (CS = 103). Considering the relationship of “measurement usability” with “technical adequacy” (r = 0.380) and “own-demand” (r = 0.694), the usability of applications increases when “technical adequacy” and “demand” are ensured. Considering the relationship between “monitoring app usability,” and the categories “adequacy” (r = 0.405, β = .247), “demand” (r = 0.688, β=.233), and their subcategories, more “adequacy” and “demands” are required to use the monitoring application than other applications. Furthermore, the stronger correlations of the categories “examination usability” (r = 0.424, β = .289) and “monitoring usability” (r = 0.405, β = .405) with “app adequacy,” compared with the other categories, including the subcategories, suggest that the design of these applications is not functional or they are not used effectively due to insufficient training.
Furthermore, the “treatment/care usability” (r = 0.724, β = .393) application requires more “demand” than the more experienced “counseling” (r = 0.639, β = .157) and “support” (r = 0.657, β = .215) applications. The “adequacy” (r = 317) relationship of the “treatment/care usability” application was lower than that of the other applications. Similarly, more “demand” was required for “support usability” (r = 0.657, β = 0.215) compared with “adequacy” (r = 0.314). Thus, these applications can be used without needing too much “adequacy” under high “demand.” similarly, more “adequacy” is needed for “physical examination usability” (r = 0.424, β = .289) compared with “demand” (r = 0.556).
The “operation usability” application was less related to “adequacy” (r = 330) and “demand” (r = 639) than the other applications. However, the values of “quan-usefulness” (r = 317), “qual-usefulness” (CS = 44), and “experience” (M = 13) were low. Thus, the relationship between the usability of “operation usability,” and the categories of “demand” and “adequacy” was weaker than that with other applications. The same is true for “quan/qual-usability” and “experience.” Professional statements also supported this inference.
In the multiple regression analysis, the variables significantly explained the models (p < 0.001): “app type demand” model: Care β = 0.39, monitoring β = 0.23, consultation β = 0.16, and support β = 0.22, (70%); “usefulness” model: demand β = 0.75 and adequacy β = 0.11, (64%); “demand” model: usefulness β = 0.75 and adequacy β = 0.10, (64%); “adequacy” model: demand β = 0.24, usefulness β = 0.25, and education β = −0.24, (25%); and “app type adequacy” model: examination β = 0.29 and monitoring β = 0.25, (22%).
Discussion
Overall, the results of this study reveal that the majority of healthcare professionals used at least one application from each type of RHS either informally or formally. The most experienced branches are internal medicine, cardiac, pediatrics, infection, and orthopedics. Internal (surgical) branches come to the fore in terms of application experience (application recommendations) and current (new) applications. Illustrating the constant pursuit of progress in healthcare, the growth curve of robotic surgery points to innovation. 48
While application recommendations are mostly for support, measurement, and evaluation applications, applications which employees have more experience of using, such as “Consulting,” “Support,” and “Monitoring,” are perceived to be more usable. Unlike the current study, a study conducted with physiotherapists revealed that the applications were mostly used in monitoring, treatment, and evaluation. 11 In another study, doctors recommended support applications such as receiving test results, scheduling appointments, and medication reminders to their patients; however, they were reluctant to recommend applications such as test result evaluation, video conferencing, and remote monitoring of vital parameters.42,49
İn this study, professionals find applications that they actually used to be useful. Furthermore, professionals want to use more technical applications if the necessary conditions are met. Similar results were reported by a study conducted in Korea: Those with experience in distance therapies exhibited a significantly higher willingness to continue participation throughout the institutionalization process. 50 Early adopters of telemedicine applications may influence others to try them. A strong recommendation intention indicates a positive experience and successful adoption. Recommendation intention emerges as an important outcome variable influenced by a number of psychological and motivational factors, such as “performance expectancy” and “effort expectancy.” 23 Medical practitioners’ awareness and positive attitudes play an important role in the effective functioning of RHS systems. 51 An interesting finding of this study is that professionals’ self-use demand exhibits a stronger association with application usability than patients’ demand. The relationship of education and experience with applicability, which is frequently emphasized in the literature; must be examined from different perspectives. As such, rigorous, in-depth research should be conducted on professional regulations for RHS content creators, RHS operators, RHS engineers, RHS safety experts, virtual health assistants, hologram doctors, and so on, as a professional function or as new professions.
The applicability categories in this study exhibit moderate to high correlations. All three applicability categories significantly explain each other in the multiple regression models. Educational status, a demographic factor, is also included in the “Adequacy of Requirements” model. The adequacy of requirements for applicability is lower for older adults than for the youth, for internal/surgical branches than for mixed branches, and for physicians than for other professionals. Furthermore, those with higher education perceive “staff number” as low score and “care usability” as high score . Thus, technical and human resources must be carefully managed for RHS applicability. Insufficient integration resulting in unnecessary repetitions increases workload and affects user demand. 38 Healthcare managers should also consider demographic influences when making assignments. For example, because they interact with technology more, the younger generation may be more aware of the benefits of technology or new applications than the older generation.
User participation in health technology design is very important for application usability. The development of RHS systems requires a multidisciplinary approach with patient and healthcare professional participation. 35 By considering user suggestions during the design process, user-friendly applications can be developed as technical solutions can be incorporated in advance. 52 For instance, a study evaluating access to remote treatment revealed that, while the majority of individuals have access to digital devices, the system used for remote therapy does not meet the minimum requirements. 53 For remote monitoring application in chronic patients, important features such as data storage and transmission, user interface, and alarm have been successful. Furthermore, the system design allows the incorporation of data sources other than medical data, such as temperature, humidity, or pollution level. These data can improve predictions and may be especially important for chronic patients with respiratory problems. 9 In another study, healthcare providers indicated that adopting, scaling, and sustaining technology-enhanced nutrition care models benefits patients, clinicians, and healthcare overall. 54
Potential barriers to RHS adoption include resistance to change, resistance to use, perceived risk, status quo bias, work-related issues, quality of care concerns, and organizational challenges. 55 System complexity, lack of user training, lack of system integration, security and privacy, inadequate technical support, and inflexibility of systems in use have been cited by healthcare professionals. 56 Resistance from healthcare providers may arise from lack of training, limited clinical knowledge, low levels of staff participation, reduced productivity, lack of required routines, communication problems, unplanned and ineffective implementation, limited resources leading to infrastructure deficiencies, technology usability challenges, design and software barriers, integration issues with other information technology systems, fears related to system effectiveness or performance risk, and doubts about the clinical and cost-effectiveness of the system. 55
The metaverse could offers a tremendous opportunity for RHS applicability. The metaverse comprises various virtual components that represent embodiments of their physical counterparts, such as digital avatars and three-dimensional virtual environments. Technologies that enable the metaverse include augmented reality, AI, distributed computing, digital twins, and telecommunications. 15 These components can enhance the quality of remote interaction.
Lack of demand for use of RHS may be due to privacy and data security concerns. 15 To address security concerns; Real-world evaluations such as “wireless hacking test” are carried out in environments where IoT devices are used in RHS applications; safety precautions must be taken. 57 To practically ensure the scalability and security of RHS applications; security models suggested for smart application design can be used. 58 Studies have suggested security solutions such as rigorous authentication processes, data encryption, resistance to attacks, and continuous monitoring, emphasizing the need to support patient autonomy, ensure confidentiality of data, and maintain equal access to healthcare in the context of IoT communication security. Measures against the ever-evolving threats in smart healthcare environments must be incorporated into application design. 43 These privacy-enhancing strategies will align RHS applications with global data protection standards in ensuring the security of patient data.
Overall, combining current study findings with those from the literature, the critical features needed in an RHS application are illustrated in Figure 9. Although the features vary depending on the type of service, most features are indispensable in a digital application owing to the connection between the services. Another crucial inference is that design requirements demonstrate the importance of the digital health engineering field.
Critical application feature.
The strategic gaps in smart healthcare applications are “legal regulations,” “economic factors,” “user behavior habits,” “politics and culture,” and “environment and technology.” 24 Studies have emphasized the integration of political, technological, 59 organizational, 60 and cultural 61 aspects, as well as the development of new applications in these directions to increase the socioeconomic benefits in RHS delivery. The current study's proposed future version of RHS includes features such as multi-faceted translation, multiple and flexible integration capabilities, personalization, alternative options, sensor detection, and advanced interaction. These features can significantly increase the applicability of the RHS from a socio-demographic perspective.
In summary, the extant application features experienced related to a RHS operate within a limited framework. Furthermore, the system is not useful enough, applications are not integrated and used officially, and application features need to be significantly improved. Several applications with high application potential exist for every branch and service type, and are even used in some institutions. Application recommendations are mostly related to integrating existing applications into the corporate system either directly or through development. Nonetheless, there are interesting suggestions for future innovation. To move forward in this regard, the necessary conditions and user demand must be met. Table 9 presents meta-inferences and sample applications regarding application recommendations based on RHS application types.
Remote health application: examples of useful RHS recommendations—existing, developable, or innovative applications.
Physical examination
The application type wherein usability is most dependent on adequacy is physical examination. However, compared with other applications, a lower demand is required for its usability. The use of physical examination was mostly recommended in the outcome evaluation phase or for greenfield patients (Appendix 3). Remote examination can be used effectively in a system that enables support, measurement, monitoring, and evaluation applications. For example, applications that recommend examination or treatment decisions can be developed by adequately detecting the patient's condition. These decisions can be verified by remote examination. Furthermore, more effective results can be obtained when using applications with advanced technology.
Studies have exhibited the potential of new technology to remotely enable individuals to experience the sensations of touch, sight, and sound, which are indispensable for physical examination. 35 Another study comparing simultaneous remote and in-person musculoskeletal examination using augmented reality, tactile sensors, and three-dimensional imaging found that the evaluations matched. 62 In another study comparing remote and face-to-face application of heart and lung auscultation, remote application was found to be an acceptable alternative to face-to-face application. 63
Measurement
Compared with consulting practice, the usability of measurement application depends more on technical adequacy and health professionals’ own demand. Concerns about the reliability of applications may impact professionals’ demand for RHS use. A study on biometric measurement and monitoring using a smartwatch found that patients were satisfied with the application; however, technical problems reduced usability. 64
In the current study, suggestions were reported for measurements such as sound, hearing, touch, vision, rhythm/motion, pressure, and temperature using sensors. Other studies have shown that remote measurements such as height/weight, 65 auditory brainstem response testing, 66 heart rate, 67 or blood pressure, 68 among others, are feasible.
Evaluation
The evaluation application is the least related application to “adequacy” and second least related application to “demand.” However, it can be considered the third most useful application. This study suggests that, medical device data should be transferred to the RHS information system for better evaluation. Here, health status-specific screening and assessment applications can be used. Additionally, applications that evaluate functions, such as sleep, nutrition, movement, respiration, circulation, excretion, and emotionality, can be integrated into the system. Innovative suggestions for malignancy prediction using noninvasive chips or digital twins have also been presented.
Some applications on evaluation have been reported in the literature. For instance, patients at risk of mortality can be identified by the remote assessment of heart failure symptoms. 69 Another study found that “prehospital assessment using commercial mobile phones with fifth-generation wireless communication technology is feasible and reliable during ambulance transport in urban areas.” 70 Additionally, the telemedical approach can significantly reduce the wait time for specialist evaluation in a real-world setting. 71
Treatment/care
Compared with other applications, less adequacy and more demands are needed for usability of treatment/care application. Improved versions of applications exist, such as medication use, mobilization, diabetes, circulation, nutrition, wound care, excretion, stress, sleep, and respiratory management, which are recommended to be used within the scope of remote care-treatment. Many applications are actually used unofficially. Health professionals believe that their use will be beneficial. Moreover, those with higher education perceived “Care Usability” as high.
A scoping review of distance physiotherapy found remote therapy to be safe, feasible, and acceptable to patients, and more cost-effective than face-to-face therapy. 72 Another study found a positive usability evaluation of a digital self-management app for depression. 73 Furthermore, the use of the remote health program was associated with significant clinical improvement in anxiety and depression. 74 Additionally, studies have concluded that remote health intervention programs can be applicable, safe, and effective in the rehabilitation process of neurological diseases. 75
Operation
Operation is the fourth application that the participants perceived as related to the “demand” and “adequacy” categories. This application is the least experienced but the second most usable application. In applications involving direct intervention, security or uncertainty may affect applicability more than other factors. For example, contemporary robotic surgery systems face challenges such as security, privacy, reliability, latency, and costly impacts of blockchain-based storage. 76
Developments such as augmented reality, AI, machine learning, integration of imaging and visualization technologies, improved precision and dexterity of robotic arms, robotic design allowing flexible surgery, tactile feedback, and sensory enhancement, are increasing the potential for robotic surgery application. 48
Despite reservations regarding using RHS in surgical branches, important suggestions have been made for operative patient groups related to planning, operation simulation, robotic surgery, three-dimensional imaging, or a common platform, in addition to pre- or post-procedure follow-ups. Considering these suggestions, a common platform can be created, supported by AI and augmented and virtual reality technologies, wherein surgeries performed throughout Turkey can be monitored or relevant experts can be involved when necessary. The technology to simultaneously connect multiple consoles to perform a single procedure is especially beneficial for patients with complex medical conditions that require a multi-specialty approach. Experts from different disciplines can connect from different locations and collaborate on the same patient in real time. 77 The application system may have a feature that allows operation planning and simulation on a person-by-person basis. Such an application not only supports surgeons but also helps assistants obtain experience from cases they have not encountered before. AI technology can help create visual animations and aid surgeons in understanding what they cannot see. 78 Additionally, decision support applications can be used during the operation, providing both visual and informational support on a transaction basis.
Monitoring
In terms of usability, the second most dependent application on “Adequacy” and “Demand” is “Monitoring.” This application is also the third most experienced and most usable application. A study on monitoring, diagnosis, and treatment applications revealed that wireless technology is mostly used in the monitoring area. 79 Based on the results of the current study, the monitoring application must be strengthened with supporting applications. For instance, the “patient wristband application integrated with the RHS system, with biometric measurement, tracking and analysis features” suggested in this study may be an interesting application.
Similar to the results of this study, other studies have also recommended using health condition-specific applications. For instance, in a study conducted with heart failure patients, the monitoring application was found to be effective. 80 Similar results have been obtained for patients with chronic9,81 and psychotic issues, 10 Parkinson's, 64 and so on.
Counseling
The usability of the consulting application is higher than that of other applications when sufficient conditions and patient demand are provided. Participants suggested that the most used consultancy application should be done through the formal application and one with more technical features. Previous studies support this conclusion. For example, remote consultation has been shown to be important and reduces the number of patient visits, especially when isolating those with infectious diseases. However, the current RHS system must be improved with support applications.40,82 Another study found that chatbots developed for medication counseling supported healthcare professionals. 83 Currently, counseling is generally performed via one-on-one meetings. In informal applications, it can also be done through a portal with certain groups of consultants or counselee. Consultation groups practice (consultant groups) is being carried out informally among health professionals usually using WhatsApp. Participants stated that exchanging ideas on a subject is useful and frequently used in groups comprising professionals or experts from different branches. The application is especially effective in using multiple minds and reaching a common conclusion in evaluating critical situations. The official implementation system should be developed considering these situations. As an application feature, transaction-based communication styles should be created.
Support
An application whose applicability is secondarily dependent on patient demand is the support application. This application is the second most experienced and most usable application; compared with other applications, less adequacy and more demands are needed for its usability. Notably, most application recommendations have been made for this application. This application type is widely discussed, especially in digital health literature. Nonetheless, it has a wide application and development potential. For instance, a study evaluating the support practices used by the Turkish Ministry of Health found that the most used E-Nabız service was finding out the examination results. 84 Similar applications were recommended to patients by doctors. 42 In this study, application recommendations were made for decision support, education, information, guidance, preventive health, and patient acceptance. Thus, there is a wide-ranging potential for information applications. Scholars can examine digital content production on a selected topic, such as exercise, patient information, and obtaining consent. Content does exist on most subjects; however, this the content should be produced by professionals, or at least revised and updated. This also reveals “digital health information” content production as a new professional function. Thus, virtual health assistant application based on AI 85 can be more effective than existing applications.
The most striking code in support applications is RHS environments. In the literature, location designs such as smart hospital, house, city, building, apartment, outdoor, or mobile health-based design have been mentioned for RHS.33,79,86 Considering the recommendations made in this study, RHS environments can be in certain locations such as a health institution, home, or in the city/intercity. Certain applications such as measurement, interview, monitoring, treatment, or operation must be possible in the environment. For this, application features must be designed very well. Especially for treatment and operation applications, adjustments such as device, medication, and environment settings should be made remotely. The recommendations in this study support and expand the literature on future innovation.7,15,16
Based on these recommendations, voice recognition can be especially used in support applications for various tasks and processes, such as note taking, reporting, transaction recording, and commanding. This technology is used in smart hospitals, including electronic medical document transcription, pathological voice recognition, and medical process optimization through human–medical equipment interaction. 87 Furthermore, remote support and inspection applications with sound, image, sign, and text features can be designed with a feature that enables communication functions, such as messaging when necessary, and performs different converting different functions, such as converting voice to text.
Integration
Integration should include user, data, device, application, and system. The participants emphasized that, the applications can be integrated with E-Nabız or hospital information systems, all necessary data can be transferred to the system, and access to the system can be provided through a common application. In this context, the management of big data is crucial to ensure health data standardization, quality, and integrity. To ensure this, a general health database should be developed at the national level to which other applications and standardized health institution databases can be connected. Moreover, applications should be accessible within the framework of the needs and authority of the institution and application users, and digital health identities should be capable of being integrated into different health systems.
Furthermore, the integration of frequently used personal devices, such as smart watches and mobile phones, into the general system is among the notable suggestions. It is stated by the participants that, the applications used must have a mobile version and be installed on the mobile phones of users such as patients or healthcare professionals. However, paradoxically, mobile phones, one of the wearable-portable devices that are indispensable for RHS use, do not have a specific standard as an RHS device and are often used informally. This issue must be addressed. An in-depth examination of the integration of both existing applications and digital technologies that will enable the use of these applications into the RHS system is needed.
Limitations
This study has some limitations. First, it was conducted on a sample from a university hospital within the Turkish healthcare system. Nonetheless, besides minor institutional differences, the results provide useful information suitable for a regional or international audience as health services are universal, and especially standardized in hospital settings. Additionally, the contributions of academic health professionals who have worked in various institutions are valuable.
Second, this study considered only some feasibility factors owing to the nature of the study. Other dimensions, such as social, economic, cultural, politics, law, and security, are important in terms of feasibility and each may require detailed investigations. Some of these dimensions were added in the initial versions of this study's questionnaire; however, based on the expert evaluation, it was decided that the dimensions would be a separate study subject owing to their scope and sensitivity. For example, finances are under the control of the institutional owner and management, especially in hospitals. As such, this factor should be examined more comprehensively by healthcare service managers, providers, and financiers. Furthermore, during the trial phase, healthcare professionals perceived these issues as technical aspects and described them as subjects they had secondary knowledge about. Similarly, the demand factor was limited to the primary users of healthcare services. Politicians and supplier companies have quite important roles or expectations in the use of RHS. Finally, the security dimension should be studied in more detail to reveal the different risks of each type of application. An up-to-date measurement tool can be developed based on the type of application by also benefiting from the features outlined here regarding the functionality of the applications.
Conclusion
This study provides strong inferences regarding the characteristics of RHS applications. Thus, it contributes unique insights regarding the potential for RHS implementation. The results reveal that the adequacy of requirements, patient–professional demand, and usability categories are important for RHS applicability. Professionals find the applications they have experienced useful and want to use other applications as well if the necessary conditions are met. Interestingly, patient–professional demand has a greater impact on the usability of applications compared with the adequacy of requirements. Additionally, the professional's own-use demand for RHS usability is more effective than the patient's demand. Notably, our results are obtained both from quantitative and qualitative data which consider the needs and interests of professionals. Consequently, our conclusions regarding the application features are robust and valuable. Based on the results, we suggest that, for a hospital of the future, RHS application should have at least these critical features. For instance, as defined in the specifications, virtual assistants and avatars should be specific to patients and professionals. Furthermore, we provide interesting examples of existing, improvable, or innovative application suggestions that can be integrated into the RHS system, such as meta-inferences based on application type, which are derived from combining professional recommendations.
Moreover, education and experience significantly affect applicability. This result, which is frequently emphasized in other studies, draws attention to a missing issue in terms of adaptation to the digital age: For RHS applicability, professions or professional functions should be updated according to the newly emerging application potential. Furthermore, differences between demographic groups are mostly related to the adequacy of requirements. Hence, healthcare managers should consider demographic factors, such as age, education, profession and experience, when planning resources and staff. Similarly, application developers should consider demographic and applicability categories along with the features that an RHS application should have. For example, patient–professional application users should have usage options that consider their age, education, profession, health status, and so on. Given the substantial demand and necessary conditions, tremendous potential exists for developing and inventing usable applications.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076251315786 - Supplemental material for Evaluating remote healthcare practices: Experiences and recommendations of healthcare professionals on smart applications
Supplemental material, sj-docx-1-dhj-10.1177_20552076251315786 for Evaluating remote healthcare practices: Experiences and recommendations of healthcare professionals on smart applications by Fadime Baştürk, Arif Osman Tokat, Osman Öztürk, Çiğdem Kader and Levent Işikay in DIGITAL HEALTH
Supplemental Material
sj-docx-2-dhj-10.1177_20552076251315786 - Supplemental material for Evaluating remote healthcare practices: Experiences and recommendations of healthcare professionals on smart applications
Supplemental material, sj-docx-2-dhj-10.1177_20552076251315786 for Evaluating remote healthcare practices: Experiences and recommendations of healthcare professionals on smart applications by Fadime Baştürk, Arif Osman Tokat, Osman Öztürk, Çiğdem Kader and Levent Işikay in DIGITAL HEALTH
Footnotes
List of abbreviations
Acknowledgements
We would like to thank the hospital's head nursing department for contributing to the implementation of the surveys during the data collection process. We would also like to thank the healthcare professionals who participated in the study and shared their opinions, ideas, and experiences. We would like to thank Editage (
) for English language editing. We would also like to thank the reviewers and editorial team for their comments that contributed to the quality and expressiveness of the study.
Contributorship
F.B.: conception or design of the work (65%), data collection (65%), data analysis and interpretation (65%), drafting the article (65%), and critical revision of the article (65%). A.O.T.: conception or design of the work (20%), data collection (20%), data analysis and interpretation (20%), drafting the article (20%), and critical revision of the article (20%). O.Ö.: conception or design of the work (5%), data collection (5%), data analysis and interpretation (5%), drafting the article (5%), and critical revision of the article (5%). Ç.K.: conception or design of the work (5%), data collection (5%), data analysis and interpretation (5%), drafting the article (5%), and critical revision of the article (5%). L.I.: conception or design of the work (5%), data collection (5%), data analysis and interpretation (5%), drafting the article (5%), and critical revision of the article (5%). Final approval of the version to be submitted: all named authors have read and approved the final version of the manuscript.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available due to limitations of ethical approval involving the participants’ data and anonymity but are available from the corresponding author on reasonable request.
Consent for publication
The purpose of the research and confidentiality of the data obtained were explained to the participants, and their informed verbal and written consent was obtained.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethics approval and consent to participate
The Yozgat Bozok University Ethics Committee approved the research on 21 September 2022, with reference number 36/18. The purpose of the research and confidentiality of the data obtained were explained to the participants, and their informed verbal and written consent was obtained.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was supported by Yozgat Bozok University Scientific Research Projects Commission under the code THD-2023-1093.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
