Design and validation of a preliminary instrument to contextualize interactions through information technologies of health professionals

Abstract

The communication of e-Health has been transformed with the advancement of information technologies, therefore it is feasible to carry out studies in the context of health professionals’ interactions. Objective: This study aimed to design and validate a preliminary questionnaire to investigate the context of the communications of health professionals through information technologies considering three significant dimensions. Method: The stages provided by Hernández Sampieri guided the building, validation through Cronbach’s alpha and factorial analysis. The questionnaire was applied to 43 participants who simulated health professionals. Results: We obtained an instrument that includes a demographic data section and 20 items distributed into three factors. Internal consistency reliability with Cronbach’s alpha values generally of 0.848 and higher than 0.811 was obtained in each dimension. Kaiser-Meyer-Olkin’s measure of sampling adequacy was regular, with 0.781, and Bartlett’s test of sphericity was significant (p < 0.001). Conclusion: It is necessary to apply in real-world environments to reaffirm the results obtained.

Keywords

health professionals information needs information technologies interactions questionnaire

Introduction

In most clinic cases, the factors associated with the health condition of hospitalized patients, the lack of information, and how it is provided represent a source of uncertainty for their relatives. The lack of clarity and the ambiguity caused by the poor communication of the health personnel is a serious problem.¹ Thus, improving the communication between health personnel and the relatives of hospitalized patients may help the relatives to develop effective mechanisms and reduce negative feelings.² Therefore, effective communication in this scenario is essential in the modern care of hospitalized patients.^3,4

In this sense, Information Technologies (IT) contribute significantly to critical levels of medical attention by reducing the communication barriers between health professionals and relatives of hospitalized patients.⁵

In this sense, the literature evidences the studies conducted on the topic. However, the research mainly focused on evaluations of implementations of communications solutions with IT,^6–8and there are studies of the necessities of the relatives of patients in different health areas.^9–14

To study the context of the interactions of health professionals with the relatives of hospitalized patients through information technologies, considering the use of information technologies in the provision of the service, the information needs, and their willingness to carry out the interactions through new forms of communication guides developers and researchers to design, develop and implement eHealth^15,16 services in hospitals. However, no similar instrument was found for this in the public literature consulted.

Therefore, this paper aims to develop and validate a questionnaire that allows studying the context of health professionals’ interactions using Information Technologies from significant dimensions: information technologies, information needs, and interactions.

We carried out a pilot project simulating health professionals to conduct its validation.

Methodology

Study design

We conducted a two-stage descriptive transversal study.¹⁷ In the first stage, the design and generation of the questionnaire were realized on Hernández-Sampieri.¹⁸ In the second stage, the questionnaire was applied in a pilot study to a sample that simulated health professionals^19–21 for your validation (data analysis) of the internal consistency in general, and each one of the dimensions was carried out using Cronbach’s alpha coefficient²²; the adequacy of the factor analysis was evaluated using the Kaiser-Meyer-Olkin (KMO) and Bartlett’s sphericity tests²³; and for the construct validity of the questionnaire, an exploratory factorial analysis was carried out^24–27 using the extraction method of principal component analysis²⁸ with orthogonal rotation through the varimax method.^29,30

The design and generation of the questionnaire consist of the following phases:

Phase 1: Fundamental Redefinitions

Notably, the study was conducted during the severe acute respiratory syndrome Coronavirus 2 (SARS-CoV2) pandemic.³¹ Therefore, the pandemic aroused several issues related to psychological distress, Personal Protection Equipment (PPE) shortage, social stigmatization, and poor coordination and management provided by health professionals.^32–34 Likewise, the pandemic caused health professionals to be inaccessible.³⁵ Furthermore, in its practice, the medics also experienced poor communication and stressful situations.³⁶ Thus, all these factors were considered to conduct the study in investigation projects.³⁷ In this sense, the study focuses on the following variables: the use of Information Technologies, information needs, and the interactions of health professionals, which are affined to contextualizing the interactions.

Previously, without an available sampling frame, we decided to simulate health professionals to analyze the behavior of the data that throws off the application of the tool.³⁸ In this regard, a successful simulation of users was based on the compromise of the participants in their roles and the proper connection among those involved with the searched purpose,¹⁹ and with a simulation inspired on Sir William Osler (1849-1919) widely considered as the ideal medic for the humanism that contributed to the exercise of the medicine.²⁰

Phase 2: Revision focused of the literature

Thus, a bibliographic search was carried out on the databases: ACM, IEEE, PubMed, Elsevier, Springer, and Web of Science. The English terms employed in the investigation were: Interaction or communication, Information Technologies, relatives or parents, and hospitalized patients in the published papers from January 2016 to June 6th, 2021.

The results of the search yielded works that counted with a measurement instrument to the defined variables in the study, such as the case of the work entitled, Use of information and communication technologies by Argentine pediatricians conducted by Dr Guillermo Goldfarba,³⁹ of information needs of the relatives of patients in base to the article The Needs Questionnaire of Relatives of Intensive Care Patients (CCFNI) brief version: adaptation and validation in the Spanish population^13,14 and for the interactions, the work that places as manifest the evolution of information technologies in medicine that probably will incorporate too quick changes positively affecting the actual condition of the communication among the health professionals, the patient, and the family.¹⁵

Phase 3: Identification of the variables to measure and its indicators

Based on the reviewed papers, a questionnaire was created to assess demographic data (sex, age, residency area, work area), followed by IT use (devices connected to the internet), information needs (information provided to the relatives of hospitalized patients) and interactions (willingness to interact it an IT system).

Phase 4: Key decision makingSince there is not an appropriate questionnaire in the existing literature, a new instrument was created to fit the variables of this study.

Phase 5: Construction of the instrument

Table 1 presents in a resumed way process of the creation of the instrument.

Table 1.

Review of the creation process of the instrument.

Version	Dimensions	Items	Sample	Observations
1	• Demographic data	41 and a section of general comments	60	• Diverse types of answers complicated the calculus of the reliability and the validity of the instrument
	• IT usage
	• Information needs
2	• Demographic data	22	385	• We added a third option of the answer to the item of sex and residence
	• IT usage			• It deleted the dimension information needs
	• Information needs			• It is incorporated the dimension interactions
	• Interactions			• There were realized corrections of redaction in items
3	• Demographic data	17	394	• We reduced redundant and unnecessary items
	• IT usage			• We realized redaction corrections of in items
	• Interactions			• Stable reliability and validity
4	• Demographic data	21	263	• It was incorporated the dimension
	• IT usage		263	• Item 20 did not show behavior fairly
	• Information needs		393	• Stable reliability and validity
5	• Interactions	20	40	• We grouped items in the three theoretical dimensions
				• Stable reliability and validity
				• An instrument according to the purpose of the study

Source: Self-made elaboration.

Finally, the instrument consists of an initial question (in which the participant decides whether to participate in the interview) and four sections: Demographic Data with five questions, IT Use with four questions, Information Needs with four questions, and Interactions with seven questions. The answer was consistent with the Likert scale, which provides five possible answers (5 = Always, 4 = Almost every time, 3 = Most of the time, 2 = Only sometimes, 1 = Never) (see Table 6 in Appendix 1).

Phase 6: Pilot study

The instrument was applied to a sample of 40 simulated health professionals,^19,20 via internet³⁸ through Google Forms.

Phase 7: Elaboration of the final version of the instrument or system and its application and interpretation process.

Finally, the clarity of the vocabulary was revised and adjusted to fit a simple and understandable language. The final version includes the title, identification, abbreviations, objectives, questions, and acknowledgments.

Data collection

It was carried out a pilot study, to validate the questionnaire in June 2022. Fifty information technology engineering students from the Polytechnic University of Victoria were invited by email to do this, where the study was explained to them and what they would answer simulating^19,20,40 the role of “health professionals” with the situation “communication with relatives of hospitalized patients considering information technologies used, the information provided and willingness to interact through a new information system.” Seven students decided not to participate. The survey was applied online to increase its accessibility.^38,41

The survey data was downloaded from Google Forms to Microsoft Excel 2016, where it was analyzed and coded. Three of the questionnaires were deleted for needing to be completed. The sample was close to 40 participants), (see Table 2). Finally, the coded data were processed using the IBM Statistical Product and Service Solutions (SPSS) version 25.0.

Table 2.

Demographics of participants.

n	Race/Ethnicity	Gender	Ages
28	Hispanic	Male	18-21
12	Hispanic	Female	18-21

Source: Self-made elaboration.

Data analysis

The reliability of the instrument and subscales were evaluated through Cronbach’s alpha^{18,22,42–44}. In addition, the paper classifies the results of the latter method.⁴⁵ Thus, Cronbach’s alpha minor or equal to 0.4 reveals weak reliability, from 0.41- 0.59 regular, 0.60 - 0.74 good, and equal or higher than o.75 means excellent reliability.⁴⁶

The Kaiser-Meyer-Olkin (KMO) test was employed to measure the adequacy of the data⁴⁷ cited in⁴⁸ to validate the construction of the questionnaire. The results indicate different results with the following guidelines: 0.00 - 0.49, there is no factor, 0.50 - 0.59 is Miserable, from 0.60 - 0.69 is Mediocre, from 0.70 - 0.79 is Regular, from 0.80 - 0.89 is Meritorious, and finally from 0.90 – 1.00 is Excelent^49,50 referred to in.¹⁶ Likewise, it was used the Bartlett test of sphericity.

Therefore, an exploratory factorial analysis was conducted^24–27 to the data using the method of extraction of analysis of principal components²⁸ cited in⁵¹ with orthogonal rotation through the varimax method.^29,30

The factorial loads over 0.3 are considered moderately high, but the value is preferred to be superior to 0.4, and those higher or equal to 0.6 are considered significantly high.⁵²

Ethics

Ethical approval was received from [details omitted for double-anonymized peer review] and [details omitted for double-anonymized peer review] of the [details omitted for double-anonymized peer review] before conducting the interviews. The potential participants were informed verbally that the questionnaire did not consider personal information or sensitive aspects of the behavior and that we emphasized that the participation was voluntary and that the participant was free to leave the research at any moment. The questionnaire was applied to the volunteers who gave their verbal approval.

Results

The questionnaire was entitled “Survey of Interactions Through Information Technologies for Health Professionals” and abbreviated as SITITHP, followed by the presentation and instructions on how to fill the document. Then, the demographic data is described briefly. Likewise, information technologies, information needs, and interactions are also displayed (see Table 7 in Appendix 1).

The Cronbach’s alpha of 0.848 shows excellent reliability for the questionnaire⁴⁵ and increases to 0.889 after eliminating item 20. The trustworthiness of the dimension obtained a Cronbach’s alpha of 0.812 for the use of IT, 0.916 for information needs, and 0.743 for interactions. After eliminating item 20 interactions, the result is 0.810.

On the other hand, the correlation is high among the items of the same dimension and lower with the rest that conforms to the rest (see Table 3). In the information dimension technologies usage, the relation among items is higher than 0.508, except for the relation of item 10 with item 7 is 0.219 in information needs over 0.695, and for the interactions are over 0.295, except with the relation of item 19 with item 17 that is of 0.186. Relations of Item 20 are inferiors, whereby were eliminated. Concerning the validity was obtained a regular value of 0.781 for the Kaiser-Meyer-Olkin (KMO) and; was significant (x² = 345.794, gl = 105, Sig. < 0.001). The results present fair values for realizing the factorial analysis in both cases.

Table 3.

Matrix of correlations.

Matrix of Correlations
	P7	P8	P9	P10	P11	P12	P13	P14	P15	P16	P17	P18	P19	P20	P21
P7	1.000
P8	0.592	1.000
P9	0.598	0.688	1.000
P10	0.219	0.530	0.509	1.000
P11	0.324	0.367	0.553	0.312	1.000
P12	0.278	0.392	0.506	0.416	0.786	1.000
P13	0.294	0.270	0.368	0.353	0.734	0.696	1.000
P14	0.418	0.439	0.489	0.431	0.703	0.777	0.728	1.000
P15	0.413	0.576	0.492	0.398	0.303	0.310	0.243	0.395	1.000
P16	0.307	0.402	0.425	0.376	0.472	0.356	0.620	0.434	0.646	1.000
P17	0.165	0.191	0.184	0.257	0.314	0.270	0.480	0.314	0.400	0.668	1.000
P18	0.093	0.240	0.385	0.290	0.329	0.293	0.466	0.439	0.529	0.625	0.481	1.000
P19	0.061	0.294	0.250	0.146	0.028	0.145	0.172	0.256	0.520	0.391	0.186	0.489	1.000
P20	−0.077	−0.015	−0.225	−0.452	−0.338	−0.336	−0.334	−0.295	0.151	0.010	0.033	−0.040	0.362	1.000
P21	0.070	0.179	0.001	0.166	0.092	0.108	0.323	0.209	0.306	0.434	0.330	0.327	0.296	0.198	1.000

Source: Self-made elaboration.

According to the results of the exploratory factorial analysis through analysis of principal components, they extracted: 3 dimensions with an auto value that determined 67.775% of the variance of the interactions through information technologies. The obtained factors were rotated with the varimax method that converged in 6 iterations, which permitted maintaining the independence among the rotated factors that determined the 43.333%, 13.245%, and 11.198% variance, respectively (see Table 4). The rotation with the varimax method allows us to visualize the factors with more detail.⁵³

Table 4.

Factorial analysis using principal components.

Component	Initial auto values
Component	Auto value	Variance of %	Accumulated %
1	6.067	43.333	43.333
2	1.854	13.245	56.578
3	1.568	11.198	67.775

Source: Self-made elaboration.

The value for the communalities was higher than 0.406 and higher than 0.524 in the matrix of rotated components, also showing the groupings of the theoretical dimensions (see Table 5).

Table 5.

Communalities and matrix of rotated component.

Communalities		Matrix of rotated component Component
Item	Extraction	1	2	3
P7	.581			.736
P8	.791			.860
P9	.780			.809
P10	.407			.525
P11	.829	.866
P12	.799	.840
P13	.851	.846
P14	.758	.763
P15	.755		.619
P16	.768		.753
P17	.552		.656
P18	.641		.741
P19	.528		.655
P21	.449		.663

Source: Self-made elaboration.

Discussion

In this study, a preliminary questionnaire has been designed and validated as a tool to investigate the context of interactions using information technologies of health professionals with relatives of patients in the hospital based on the dimensions of the use of information technologies, information needs, and interactions. Next, the findings according to the statistical analysis of the data were discussed.

The internal consistency of the proposed questionnaire was above 0.809 in general and for each of the dimensions that make up the instrument; however, it was necessary to delete one item because it was ungrouped from the theoretical approach, as was done in works related to the proposal and validation of questionnaires respectively.^54,55

In the use of information technology dimension, the weakest relationship with 0.219 was found between items 7 and 10; For its part, the first item refers to whether the health professional uses the Internet in their professional practice, while the second item questions whether they use SMS messages, email, WhatsApp, Facebook, Twitter, etc. In their daily professional practice. However, in the interaction dimension, item 20, associated with whether the health professional would have difficulties adapting to new technologies, presenting problems for their use, and not knowing how to deal with change, disagrees with the rest of the items of the dimension and of the questionnaire, since it is not positively related and since it is ungrouped from the theoretical approach, the above could be due to the negative form of the question’s approach, giving the possibility of writing the question in a positive way.^56,57

Two identified factors agreed with the proposed dimensions; Factor 1: Use of Information Technologies, made up of four items [7,8,9 and 10]; Factor 2: Information needs, groups 4 items [11, 12, 13 and 14] and finally, Factor 3: Interactions, was made up of items [15, 16, 17,18, 19 and 21]. This behavior of theoretical coherence between the items that integrate the factors is similar to that reported in the study completed in 2019 of the (TAQPH), Spanish version.⁵⁴

The total variance explained in this study is 67.775%, slightly above works that propose designing or developing of questionnaires related to the health sector.^58–61 Similarly, the number three dimension was identified, stated in the article on the tool for studying nurse-patient therapeutic communication was created and validated.⁶²

However, the size of the variance explained and the number of components obtained were smaller than the pilot research carried out in China, considering 25 students to adapt (S-EBPQ).⁶³

While it is true that the results revealed excellent reliability in internal consistency with a value of 0.848 Cronbach’s alpha in general,^22,45 a regular value of 0.781 for the KMO and Bartlett’s test of sphericity was significant,^49,50 it is necessary to highlight that the results participants simulated^19,20 health professionals, when the published literature evidences the use of simulation in the health sector in different ways in order to gather scientific evidence and where the objectives of projects can be extremely diverse,²¹ for example, simulated profiles of health professionals in social networks,⁶⁴ in questionnaires it has been used to generate exams,⁶⁵ validation of questionnaires through mock consultations⁶⁶ and simulation data⁶⁷; and in patient simulation⁶⁸ where a participant simulates a role and a situation,⁴⁰ in our case it was a role of “health professional” and the situation communication with relatives of hospitalized patients considering information technologies used, information provided and willingness to interact through a new information system.

Finally, it was shown that it is feasible to validate instruments in adverse conditions^17,32–37 by simulating^19,20,64 health professionals, which is necessary before applying the questionnaires in a study with real health professionals, based on where a participant simulated a role (patient) and a described situation.⁴⁰

Limitations

Limitations of the research include a small sample size of demographic characteristics that were drawn from a homogeneous population, although the validation results were satisfactory. A second limitation of the study was the simulation of the sample, and participants possibly needed to give precise and socially desirable answers, even having surveyed the severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) pandemic could have influenced their responses causing a memory or adaptation bias. Due to the above, caution is recommended since the results shown here were not definitive and should not be generalized.

Conclusion

We conclude that the preliminary questionnaire to study the interactions of health professionals with relatives of hospitalized patients is a valid and reliable instrument for this purpose; however, it is necessary to continue work for its validation with new confirmatory studies in hospitals.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical statement

ORCID iDs

José Fidencio López Luna

Eddie Nahúm Armendáriz Mireles

Marco Aurelio Nuño Maganda

Hiram Herrera Rivas

Rubén Machucho Cadena

Jorge Arturo Hernández Almazán

Appendix

Table 7.

Survey of interactions through information technologies for health professionals (SITITHP).

Presentation text

This survey is anonymous and confidential

Carefully read the instructions and the raised questions in each section and answer sincerity what was asked to you

By the present, I leave conformity constancy as respondent, and I agree that the collected data may be used for strictly scientific purposes, of research and education

Demographic data

The following questions are to collect personal and labor information anonymously

1.- Do you agree participating in this survey?

Yes

If the answer is “Yes” continue with the following question

If the answer is “No” the survey ends

2.- What is your sex?

Man

Woman

Other

3.- What is your age range?

From 18 to 29 years

From 30 to 39 years

From 40 to 49 years

From 50 to 59 years

From 60 to 69 years

More than 69 years

4.- What is your residence area?

Rural

Urban

Other

5. In what field do you exercise your profession?

Governmental

Private company

Independent

Does not exercise

Other

6. In what department do you exercise your work?

Social work

Nursing

Specialist

Plant doctor

Intern

Other

Information technologies usage

The following questions are to know the usage of information technologies in its professional practice

7.- In your daily professional practice, do you use internet?

Always

Almost every times

Most of the times

Only sometimes

Never

8.- In your daily professional area, do you use a computer, tablet, smartphone, etc.?

Always

Almost every times

Most of the times

Only sometimes

Never

9.- Do you count on internet connection in the place of work where you use these devices?

Always

Almost every times

Most of the times

Only sometimes

Never

10.- In your daily professional practice, do you use SMS messages, email, WhatsApp, facebook, twitter, etc.?

Always

Almost every times

Most of the times

Only sometimes

Never

Information needs

The following questions are for determining if the proportionated information to the relatives of hospitalized patients covers their information needs

11.- Do you give to know to the relatives the health condition of your hospitalized patient?

Always

Almost every times

Most of the times

Only sometimes

Never

12.- Do you inform the relatives how it is medically threatening to your hospitalized patient?

Always

Almost every times

Most of the times

Only sometimes

Never

13.- Do you communicate with preciseness to the relative what is medically doing to the hospitalized patient?

Always

Almost every times

Most of the times

Only sometimes

Never

14.- Do you inform the relatives of the why of the realized procedures to your hospitalized patient?

Always

Almost every times

Most of the times

Only sometimes

Never

Interactions

The following questions are to know the disposal of realizing the interactions through the usage of an information system. In the survey, it makes reference to an information system as a website, a program, or an application

15.- Do you consider it is necessary to implement the usage of an information system to speed up the access to the information and the follow-up with the relatives of hospitalized patients that you attend in a direct or indirect way?

Always

Almost every times

Most of the times

Only sometimes

Never

16.- Will it benefit you the information systems usage to the communication with the relatives of hospitalized patients that you attend directly or indirectly?

Always

Almost every times

Most of the times

Only sometimes

Never

17.- Do you think it is necessary to provide multimedia content (photos or videos) about the patient at the relatives through an information system usage?

Always

Almost every times

Most of the times

Only sometimes

Never

18.- Do you think it is necessary to realize petitions of materials (personal cleanliness, clothing, etc.) or documentation for the hospitalized patients to their relatives through an information system?

Always

Almost every times

Most of the times

Only sometimes

Never

19.- Will you use an information system with communication functions with the relatives of hospitalized patients, to those whom you attend indirectly or directly?

Always

Almost every times

Most of the times

Only sometimes

Never

20.- Is it in your interest to inform the following of the hospitalized patients that you attend directly or indirectly through an information system usage and that this notifies automatically relatives of hospitalized patients through SMS messages, email, WhatsApp, facebook, and twitter?

Always

Almost every times

Most of the times

Only sometimes

Never

Source: Self-made elaboration.

References

Caswell

Pollock

Harwood

, et al. Communication between family carers and health professionals about end-of-life care for older people in the acute hospital setting: a qualitative study. BMC Palliat Care 2015; 14: 35. DOI: 10.1186/s12904-015-0032-0.

Velandia Galvis

Bautista Espinel

Arevalo Rangel

, et al. Incertidumbre en los cuidadores de niños hospitalizados en unidades de: cuidado intensivo neonatal – cuidado intensivo pediátrico (UCIN-UCIP). Cult los Cuid 2019; 23: 256–269. DOI: 10.14198/cuid.2019.55.22.

Pagnamenta

Bruno

Gemperli

, et al. Impact of a communication strategy on family satisfaction in the intensive care unit. Acta Anaesthesiol Scand 2016; 60: 800–809. DOI: 10.1111/aas.12692.

ADIP . Familiares de pacientes hospitalizados por COVID-19 podrán comunicarse por medio de videollamadas, 2020, https://adip.cdmx.gob.mx/comunicacion/nota/familiares-de-pacientes-hospitalizados-por-covid-19-podran-comunicarse-por-medio-de-videollamadas

Robledo-Madrid

. Impacto de la aplicación de las Tecnologías de la Información (TICs) en un modelo de atención médica. REVISTA DE SANIDAD MILITAR MEXICO 2020; 73: 96–104.

Wieck

Blake

Sellick

, et al. Utilizing technology to improve intraoperative family communication. Am J Surg 2017; 213: 895–900. DOI: 10.1016/j.amjsurg.2017.03.014.

Globus

Leibovitch

Maayan-Metzger

, et al. The use of short message services (SMS) to provide medical updating to parents in the NICU. J Perinatol : Official Journal of the California Perinatal Association 2016; 36: 739–743.

Aufegger

Hà Bùi

Bicknell

, et al. Designing a paediatric hospital information tool with children, parents, and healthcare staff: a UX study. BMC Pediatr 2020; 20: 469.

Leske

. Internal psychometric properties of the critical care family needs inventory. Heart Lung : The Journal of Critical Care 1991; 20: 236–344.

10.

Lee

Park

Lim

, et al. Care needs of patients at the end of life with a noncancer diagnosis who live at home. J Nurs Res 2019; 27: e10.

11.

Padilla-Fortunatti

Rojas-Silva

Amthauer-Rojas

, et al. Needs of relatives of critically ill patients in an academic hospital in Chile. Enfermería Intensiva 2017; 29: 32–40.

12.

Ning Tan

Tiew

Shorey

. Experiences and needs of parents of palliative paediatric oncology patients: a meta-synthesis. Eur J Cancer Care 2021; 30: e13388.

13.

Gómez

Ballester Arnal

Gil Juliá

. El Cuestionario de Necesidades de los Familiares de Pacientes de Cuidados Intensivos (CCFNI) versión breve: adaptación y validación en población española. An Sist Sanit Navar 2011; 34: 349–361.

14.

Padilla Fortunatti

. Most important needs of family members of critical patients in light of the critical care family needs inventory. Invest Educ Enfermería 2014; 32: 306–316.

15.

Weiner

. Doctor-patient communication in the e-health era. Isr J Health Pol Res 2012; 1: 33.

16.

Luna

JFL

. Propuesta de instrumento para contextualizar interacciones mediante tecnologías de la información de los familiares de pacientes hospitalizados. Revista Dilemas Contemporáneos: Educación, Política y Valores 2022; 4: 3135, DOI: 10.46377/dilemas.v9i2.3135.

17.

Moreno

ALT

Vaca Morales

Martínez

CDI

, et al. Diseño y Validación de un Cuestionario para el Diagnóstico de Riesgos Psicosociales en Empresas Ecuatorianas. Cienc Trab 2018; 20: 160–168.

18.

Hernández

Fernández Collado

Baptista Lucio

. Metodología de la investigación. New York: McGRAW-HILL/INTERAMERICANA, 2014.

19.

Dávila-Cervantes

. Simulación en Educación Médica. Educación Médica, 2014: 3: 100–105.

20.

Craxì

Giardina

Spagnolo

. A return to humane medicine: Osler's legacy. Infez Med, 2017: 25: 292–297.

21.

Santomauro

Hill

McCurdie

, et al. Improving the quality of evaluation data in simulation-based healthcare improvement projects. Simulat Healthc J Soc Med Simulat: The Journal of the Society for Simulation in Healthcare 2020; 15: 341–355.

22.

Cronbach

. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297–334.

23.

Tuba Karabey

Gulturk

. A self-efficacy scale for nursing educators’ role: a Turkish validity and reliability study. Int J Caring Sci 2023; 16: 148.

24.

Harerimana

Ntombifikile

. Using Exploratory and Confirmatory Factor Analysis to understand the role of technology in nursing education. Nurse Educ Today 2020 :92: 104490.

25.

Ledesma

Pere

Tosi

. Uso del Análisis Factorial Exploratorio en RIDEP. Recomendaciones para Autores y Revisores. Revista Iberoamericana de Diagnóstico y Evaluación - e Avaliação Psicológica 2019; 3: 173–180.

26.

Gaskin

Happell

. On exploratory factor analysis: a review of recent evidence, an assessment of current practice, and recommendations for future use. Int J Nurs Stud 2014; 51: 511–521.

27.

Izquierdo

Olea

Abad

. Exploratory factor analysis in validation studies: uses and recommendations. Psicothema 2014; 26: 395–400.

28.

Jolliffe

. Principal component analysis. New York: Springer, 2002.

29.

Lara Severino

RDC

García Orrí

Parra Pérez

, et al. Consistencia interna y estructura factorial de la escala de desinhibición alimentaria a través de las contribuciones del Análisis Factorial Exploratorio con rotación Varimax, el coeficiente de correlación de Pearson y el coeficiente alpha de Cronbach. Nutr Clínica Dietética Hosp 2019; 39: 133–140.

30.

Dien

. Evaluación de PCA de dos pasos de datos ERP con rotaciones Geomin, Infomax, Oblimin, Promax y Varimax. Psicofisiología 2010; 47: 170–183.

31.

Belliveau

Horton

Hereford

, et al. Pro-inflammatory diet and depressive symptoms in the healthcare setting. BMC Psychiatr 2022; 22: 125. DOI: 10.1186/s12888-022-03771-z.

32.

Rahman Razu

Yasmin

Binte Arif

, et al. Challenges faced by healthcare professionals during the COVID-19 pandemic: a qualitative inquiry from Bangladesh. Front Public Health 2021; 9: 647315.

33.

Tatli

Simsek

Guner

, et al. Health care professionals’ views on healthcare provision during the COVID-19 pandemic: a descriptive study. BEZMIALEM SCIENCE 2020; 9: 46–54.

34.

Nyashanu

Pfende

Ekpenyong

. Exploring the challenges faced by frontline workers in health and social care amid the COVID-19 pandemic: experiences of frontline workers in the English Midlands region, UK. J Interprof Care 2020; 34: 655–661.

35.

Ellard-Gray

Jeffrey

Choubak

, et al. Finding the hidden participant: solutions for recruiting hidden, hard-to-reach, and vulnerable populations. Int J Qual Methods 2015; 14: 1189–1202.

36.

Cannarella

Jacques

Donovan

, et al. Managing difficult encounters: understanding physician, patient, and situational factors. 2013; 87: 419–425.

37.

Caron Estrada

Mattos Navarro

Barboza Meca

. Dificultades para la elaboración de artículos de investigación científica en estudiantes de posgrado en salud. Educación Médica Superior 2020; 34: 1561–2902.

38.

Ondé

Alvarado

. Reconsidering the conditions for conducting confirmatory factor analysis. Spanish J Psychol 2020; 23: e55.

39.

Goldfarb

Nasanovsky

Krynski

, et al. Uso de tecnologías de la información y comunicación por pediatras argentinos. Arch Argent Pediatr 2019; 117: 64–76.

40.

Regmi

Waithaka

Paudyal

Simkhada

van Teijlingen

. Guide to the design and application of online questionnaire surveys. Nepal J Epidemiol. 2016 Dec 31; 6(4): 640–644. DOI: 10.3126/nje.v6i4.17258

41.

Regmi

Waithaka

Paudyal

, et al. Guide to the design and application of online questionnaire surveys. Nepal J Epidemiol 2016; 6: 640-644. DOI: 10.3126/nje.v6i4.17258.

42.

Kocatepe

Kayıkçi

Saygılı

, et al. The palliative care outcome scale: Turkish validity and reliability study. Asia Pac J Oncol Nurs 2020; 7: 196–202.

43.

Sierra-Zúñiga

Holguin Betancourt

Mera-Mamián

, et al. Construction and validation of an instrument to measure knowledge of mothers and caregivers on complementary feeding of infants. Rev Chil Pediatr 2020; 9: 711–721.

44.

Tuapanta Dacto

Duque Vaca

Mena Reinoso

. Alfa de Cronbach para validar un Cuestionario de uso de TIC en Docentes Universitarios. mktDescubre - ESPOCH FADE 2017 10: 37–48.

45.

Cicchetti

Sparrow

. Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic 1981; 86: 127–137.

46.

Seyed Hossein Mohaqeqi Kamal

SJM

Sajjadi

Vameghi

Ghaedamini Harouni

, et al. Designing and validation of a workplace abuse questionnaire among child labour. Child Soc 2022; 36: 790–805. DOI: 10.1111/chso.12534.

47.

Kaiser

. An index of factorial simplicity. Psychometrika 1974; 39: 31–36, DOI: 10.1007/BF02291575.

48.

Watkins

. Exploratory Factor Analysis: A Guide to Best Practice. Journal of Black Psychology, 2018; 44: 219–246. DOI: 10.1177/0095798418771807.

49.

Beavers

Lounsbury

Richards

, et al. Practical consider actical considerations for using explor ations for using exploratory factor analysis in or analysis in educational research. Practical Assess Res Eval 2013; 18: 1–13.

50.

Abubaker Blbas

Hussein Kadir

. An application of factor analysis to identify the most effective reasons that university students hate to read books. International Journal of Innovation, Creativity and Change 2019; 6: 251–265.

51.

Ringnér

. What is principal component analysis? Nat Biotechnol 2008; 26: 303–304.

52.

Sumaya

. Using exploratory factor analysis in personality research: best-practice recommendations. Journal of Industrial Psychology/SA Tydskrif vir Bedryfsielkunde 2010; 36: 7.

53.

Wakeling

Horn

. Neuromechanics of muscle synergies during cycling. J Neurophysiol 2009; 101: 843–854.

54.

Esquivel Garzón

Díaz Heredia

. Validity and reliability of the treatment adherence questionnaire for patients with hypertension. Invest Educ Enferm, 2019; 37(3): e09.

55.

Abd

EHS

Ahram

Samir

, et al. Development and validation of a biobank questionnaire intended for the public in the arab region. Biopreserv Biobanking 2021; 19: 422–431.

56.

Shapiro G

Tatar

Dube

, et al. The vaccine hesitancy scale: psychometric properties and validation. Vaccine 2018; 36: 660–667.

57.

Vilca

Echebaudes-Ilizarbe

Aquino-Hidalgo

, et al. Psychometric properties of the environmental reward observation scale: study on its internal structure, factor invariance, and method effect associated with its negative items. Psychol Rep 2020; 125: 649–675.

58.

Beykmirza

Nikfarid

Negarandeh

, et al. Development and validation of an instrument to measure pediatric nurses’ adherence to ethical codes. BMC Med Ethics 2022; 23: 14.

59.

Ballouk

Mansour

Dalziel

, et al. The development and validation of a questionnaire to explore medical students’ learning in a blended learning environment. BMC Med Educ 2023; 22: 3045–3054.

60.

Youhasan

Chen

Lyndon

, et al. Development and validation of a measurement scale to assess nursing students’ readiness for the flipped classroom in Sri Lanka. J Educ Eval Health Prof 2020; 17: 41.

61.

Ayush

Piyush

Saikaustubh

, et al. Development and validation of a questionnaire for assessing preventive practices and barriers among health care workers in COVID-19 pandemic. IJMM 2021; 39: 200–211.

62.

Granados-Gámez

Sáez-Ruiz

Márquez-Hernández

, et al. Development and validation of the questionnaire to analyze the communication of nurses in nurse-patient therapeutic communication. Patient Educ Counsel 2022; 105: 145–150.

63.

Yin-Ping

Wen-Hui

Yi-Tian

, et al. Developing student evidence-based practice questionnaire (S-ebpq) for undergraduate nursing students: reliability and validity of a Chinese adaptation. J Eval Clin Pract 2018; 25: 536–542.

64.

Weijs

Coe

Desmarais

, et al. Effects of mock Facebook workday comments on public perception of professional credibility: a field study in Canada. J Med Internet Res 2019; 21: e12024.

65.

Higgins

Deal

Rinewalt

, et al. The utility of mock oral examinations in preparation for the American Board of Surgery certifying examination. Am J Surg 2016; 211: 416–420.

66.

Morel

Jean-Benoit

Pottier

. Interpersonal skills among fourth- and fifth-year medical students: validation of an assessment tool. Simulat Healthc J Soc Med Simulat 2018; 13: 88–95.

67.

Chung-Han

Da-Wei

Mei-Lien

. Grouped data with survey revision. BMC Med Res Methodol 2023; 23: 15.

68.

Sommer

Hiroko Fritz

Thrien

, et al. Simulated patients in medical education – a survey on the current status in Germany, Austria and Switzerland. GMS J Med Educ 2019; 36: Doc27.