Sage Journals: Discover world-class research

Abstract

Objective

Telenursing e-learning courses have been shown to enhance nurses’ skills and knowledge; however, the subjective learning experience is unclear. In this study, we identified meta-inferences to quantitatively and qualitatively understand this experience, as well as the types of knowledge gained through an e-learning course and how they are linked to each other, in order to enhance nurses’ confidence in their understanding of telenursing.

Methods

We employed a single-arm intervention with a mixed-methods convergent parallel design. We converged participants’ self-reported pre- and post-course confidence scores with their reflections on the learning experience, which were reported qualitatively as improved or unimproved. A total of 143 Japanese nurses with a mean of 20 years of nursing experience participated in this study.

Results

Among the participants, 72.7% demonstrated improved confidence in their understanding of telenursing after completing the e-learning course. The baseline confidence score was originally higher in the group that reported unimproved confidence (p < .001). Although there was no statistical difference in the usability and practicality scores between the two groups, the qualitative learning experience in these aspects differed in terms of the depth of knowledge of telenursing obtained.

Conclusions

Nurses’ quantitative confidence in their understanding of telenursing after course completion was incongruent with their qualitative perspectives of the learning experience. Nursing educators, healthcare policymakers, and other stakeholders should consider that learners’ overconfidence in their understanding of telenursing and comprehension of e-learning materials may result in their failure to develop key telenursing competencies, skills, and knowledge.

Keywords

Convergent mixed-methods design e-learning mixed-methods study nurse telenursing confidence

Introduction

E-learning defined as using Internet devices to enhance or support teaching and learning in an online community^1,2 is widely used in healthcare provider education, including nursing, to improve knowledge and performance.³ This use of e-learning increased dramatically in response to the challenges posed to the global healthcare system by the 2020–2023 COVID-19 pandemic.⁴ These challenges exposed the urgent need for ongoing academic and evidence-based program development to enhance telenursing practices, which emerged as a new nursing field,⁵ given that nursing professionals must constantly update their healthcare skills and knowledge.

Telenursing, an innovation in the field, is becoming particularly popular in Japan.^6,7 Telenursing refers to nursing activities that support the delivery, management, and coordination of healthcare services for patients and their families from a physical distance using information and communication technologies (ICTs).⁸ Telenursing results in the improvement of accessibility and affordability of nursing care for people with care needs living in remote areas.⁵ In the digital era, nurses can utilize these innovative tools.^9,10 They need to acquire new knowledge, skills, and key competencies, such as telecommunication and information risk management prior to entering telenursing practice.⁸ Although universities may be a logical place to provide such education, in Japan, for example, only 3.8% of nursing universities offer telenursing or telehealth education for undergraduates owing to limited teaching personnel, overloaded curricula, and restrictions on or absence of clinical practicum.¹¹ Ensuring adequate global and national healthcare deliveries requires immediate and comprehensive telenursing education; e-learning education creates necessary conditions to address these needs adequately.⁵ However, this has not been prioritized; consequently, the global healthcare workforce has remained unprepared.^11,12

Telenursing curricula require the collaboration of interdisciplinary educators, healthcare policymakers, and stakeholders to support safe and affordable healthcare. Owing to a shortage of qualified faculty, e-learning is indispensable.

For learners, an internal state of motivation and a confident attitude toward learning are essential for successful educational outcomes.¹³ The attention, relevance, confidence, and satisfaction conceptual model facilitates motivation for effective learning and instructional design.^14,15 According to Keller,¹⁵ the model consists of: attention: capturing learners’ interest, relevance: meeting learners’ personal needs or goals to affect a positive attitude to learning, confidence: helping learners believe that they will succeed and control their success, and satisfaction: reinforcing accomplishment with rewards.

This model is also useful for developing and evaluating the quality of teaching materials.¹⁶ Confidence reflects the basic motivation for learning, that is, the degree to which one feels competent through the learning experience.^15,17,18 Learners’ confidence-building strategies may focus on motivation, learning requirements, opportunities for success, and personal control.¹⁵

E-learning systems must be both usable and practical for learners to effectively achieve their learning goals with high satisfaction.^19,20 Thus, evaluating the learning experience is crucial for assessing the effectiveness of e-learning education.^13,21

Stemp et al.²² noted that in course assessments, it is insufficient to only quantitatively assess learning effectiveness in online education. Thus, adopting a study methodology that assesses both quantitative (QUAN) and qualitative (QUAL) data is necessary in thoroughly understanding the phenomenon. We posited that a QUAN and QUAL assessment of the e-learning experience and its usability and practicality from learners’ perspectives can contribute to improving nurses’ confidence in their understanding of telenursing and its effectiveness. Observing learning phenomena from multiple perspectives using mixed methods that integrate QUAN and QUAL data contributes to understanding and improving learning processes, adding value, and solving problems in educational research.^23,24 The paradigm that combines QUAN and QUAL approaches seeks to answer research questions and maximize the understanding of the subject of interest.^25–29

In this study, we focus on understanding nurses’ complex learning experiences, as well as the types of knowledge gained through e-learning education and how they are linked to each other, to raise their confidence in their understanding of telenursing through e-learning education.

Purpose and research question

We identified and developed meta-inferences to determine whether nurses’ confidence in their understanding of telenursing improved through e-learning courses based on their learning experiences.

We proposed the following research question: How are nurses’ learning experiences (QUAN and QUAL) linked to improvements in their confidence in their understanding of telenursing? We adopted multiple research methods to answer this question.

Methods

Study framework

We developed an assumption, intervention, and outcome framework to assess nurses’ learning experiences (QUAN and QUAL) from multiple perspectives (Figure 1): nurses interested in telenursing were enrolled (assumptions), experienced the e-learning process (interventions), improved their confidence in their understanding of telenursing, and gained satisfaction from their learning experience (outcomes).

Figure 1.

Study framework.

Design

We conducted a single-arm pre- and post-intervention assessment using a mixed-methods convergent parallel design²⁶ to maximize the synthesis of QUAN and QUAL data. The convergent design involves separate data collection and analysis of QUAN and QUAL data and subsequently merging the results obtained from both sets of data analysis.²⁶ This design provided a comprehensive picture of how nurses improve their confidence in their understanding of telenursing by integrating what they have learned through the e-learning program. The mixed-methods approach allowed us to undertake four essential operations—elaboration, generalization, data integration, and interpretation—and address the higher impacts of the study.^30,31 Its rigor facilitated the achievement of the study's objective, consistent with the study design, data collection, analysis, and integration.³² Fetters³³ and Tashakkori et al.²⁹ suggested that it is typical to collect and integrate QUAN and QUAL data to create meta-inferences as multiple perspectives based on the results of both datasets. These processes facilitate the development of insights that mono-method studies cannot provide. In our pre- and post-course confidence assessments, we explored learning satisfaction—an outcome of the QUAN and QUAL learning experience—using mixed-method data integration.

As illustrated in Figure 2, the QUAN data strand was used to examine the participants’ levels of learning interest in telenursing as a baseline (T0). The QUAN learning experience data included usability and practicality (T1 to T50), nurses’ confidence in their understanding of telenursing (T0 and T51), and their satisfaction with the learning experience (T51). The QUAN data were collected through a self-administered online questionnaire and an e-learning platform. For the QUAL data, open-ended responses to questions about the learning experience—including what knowledge had been obtained from each lesson—were collected upon completion of each of the 50 lessons via an e-learning platform (T1 to T50).

Figure 2.

Procedural diagram of the mixed-methods intervention design with data collection and convergent timing.

E-learning telenursing course

We developed a telenursing e-learning course to facilitate nurses’ acquisition of the basic knowledge and skills required to provide telenursing services. The actual course covers evidence-based telenursing guidelines issued by the Japan Academy of Home Care.⁸ The telenursing education program, which we have been delivering in person for clinical nurses since 2012,³⁴ was comprehensively expanded for the purpose of this study. To the best of our knowledge, our education program is the first to introduce comprehensive telenursing and capacity building for nurses in Japan. We re-organized the core lesson component taught in the former program and divided the program into smaller sections of approximately 15 min—the appropriate time for an e-learning lesson. An e-learning course comprising 50 lessons was created covering the following topics: introduction to telenursing and the health system; fundamentals of ICT literacy; telemonitoring and telecommunication skills; and practicum case scenarios. It included 42 lectures and eight practicum videos. There are few telenursing education experts in Japan; therefore, our team included nurse researchers who specialized in telenursing, home care, gerontological nursing, intractable disease nursing, clinical nursing, and nursing curriculum development, as well as a physician, a telehealth engineer, and a health communication expert. They discussed the program components and prepared each lesson separately. The first author, a pioneer in telenursing in Japan, assessed and modified the lessons to avoid overlap and maintain the recorded voice, slide image, and video quality. Due to the COVID-19 pandemic, an external assessment of the course was not possible. The e-learning was delivered via the Japanese e-learning platform, edulio.³⁵ The administrator controlled the course such that learners could sequentially complete lessons 1–50. Each lesson consisted of 5–15 min of video lectures highlighting points to learn to achieve the learning goals (Table 1). Participants could freely access the platform during the study period.

Table 1.

Telenursing e-learning lesson and the goal.

1. Introduction to telenursing: To understand telenursing	Telemonitoring and disease management ⑥ People with cancer
Definition, purpose, and characteristics of telenursing	Telemonitoring and disease management ⑦ People with intractable neurological disease
Healthcare system of Japan and related guidelines of telenursing	Telemonitoring and disease management ⑧ People with SARS-Cov2
Principle ethics of telenursing	4. Fundamentals and skills of telenursing: To understand the principals of telenursing
Types of telenursing	Ways of telenursing and people-centered care
Quality assurance of telenursing	Health literacy assessment for the telenursing client
History and movements of national/international telehealth	Primary, secondary, and tertiary prevention and the needs for continuous support
Healthcare insurance system of Japan and telehealth	Characteristics of client and family care needs for telenursing
Telehealth and scientific evidence	Skills and perspective of telenurse
2. Fundamentals of information and communication technology (ICT) literacy: To understand how to provide and manage telenursing and information security	Technical systems related to telenursing
	Principals of health communication
Information literacy to provide telenursing	Foundations of telecommunication and telementoring
Information security and privacy management in telehealth/telenursing	Fundamentals of nursing consultation for people with chronic conditions
Telenursing platforms	Foundation of store and forward system (e-mail)
Information and communication technology, internet of things with telenursing	Preparation to provide telenursing; informed consent before the start of telenursing
Telehealth with artificial intelligence (AI) and big data	Telenursing and nursing process
Information technology for next generation (5G) and telenursing	Telenursing and nursing record
3. Fundamentals and skills of telemonitoring: To understand telemonitoring in management of people with chronic conditions	Collaboration with interdisciplinary team and community care system
	Ending/completion of telenursing
Monitoring and assessment of psychosomatic state with telenursing	Case examples of telenursing
Characteristics of chronic disease management and the needs for telemonitoring	Case examples of telenursing
Management of measuring equipment	5. Practicum case scenarios–telemonitoring-based telenursing: To understand how to provide telemonitoring, telecommunication, and telenursing
Telemonitoring and disease management ① People with diabetes	Practice of telemonitoring and tele-observation
Telemonitoring and disease management ② People with heart failure	Proactive telecommunication; active listening, empathy, mentoring, and health consultation
Telemonitoring and disease management ③ People with chronic obstructive pulmonary disease	Practice of store and forward system (e-mail)
Telemonitoring and disease management ④ People with interstitial pneumonia	Case scenarios and telenursing process
Telemonitoring and disease management ⑤ People with aspirational pneumonia

Participants

Registered Japanese nurses who volunteered to participate were directly recruited through the study website (Kango-net) and through a flyer that the first author's institution distributed to the community. In addition, we sent a flyer to the executive board of directors of a Japanese home care organization and to the visiting nursing group, which invited them to present this opportunity to their nurses. We set no geographical limitations on the participants. We paid honorarium to the participants if they completed the course within the study period (October 2021–March 2022).

Inclusion criteria

The participant eligibility criteria included:

basic skills to access the e-learning course,

ability to read and write in Japanese, and

voluntary acceptance of all 50 telenursing e-learning lessons.

Sampling strategy

Few previous studies have reported healthcare providers’ confidence improvement pre- and post-e-learning; therefore, we referenced nurses’ competence development via blended learning. Blended learning has been reported to result in an almost 5.5% improvement in competence³⁶; assuming this prevalence, we aimed to obtain a 10% improvement in confidence regarding telenursing via e-learning. In order to obtain 80% power (1-β error) and with a 5% significance level (Type I; α error) and estimating a 10% dropout rate, a total of 139 participants were required (single arm) for the QUAN study.³⁷ In addition, sampling for a mixed-methods study must help answer the research question, and this depends on whether both QUAN and QUAL data are to be collected from the same or different individuals.^29,38 In this study, all the data were collected from the same participants. We employed a one-time convergent design to draw meta-inferences regarding whether and why learners’ confidence in their understanding of telenursing is expanded by linking the QUAN and QUAL learning experiences. As educational programs and clinical applications of telenursing in Japan are in their infancy, we did not include nurses’ previous experiences with telenursing education and practice as variables.

Data collection

Data on nurses’ gender, age, clinical experience, work status, and affiliation were collected for demographics.

QUAN

In line with the mixed-methods design, nurses’ interest in telenursing, learning experience, study outcomes, and other measurements were collected using self-administered online questionnaires at each time point (T0 to T51). The data sources are displayed in Table 2.

Table 2.

Mixed-methods data source table of the study.

Quantitative data collection; assessment scale (T0 to T51)		Framework	Qualitative data collection; questionnaire (T1 to T50)
Interest in telenursing (T0 and T51)	Original Likert scale (1 item). 1 (minimum) to 5 (maximum)	Assumptions	－
Usability (T1 to T50)	Original Likert scale (1 item). 1 (minimum) to 5 (maximum)	E-learning experience (interventions)	Obtained knowledge (experience) through e-learning each lesson (T1 to T50)	Describe what was obtained in each lesson (collected via e-learning platform)
Practicality (T1 to T50)	Original Likert scale (1 item). 1 (minimum) to 5 (maximum)	E-learning experience (interventions)
Confidence in their understanding of telenursing (T0 and T51)	Original self-inventory (1 item). 1 (minimum) to 5 (maximum)	Outcomes	－
Satisfaction with the learning experience (T51)	VAS-10^a:0 (worse) to 10 (best)	Outcomes	－

Visual analogue scale.

Note. T0, time point pre e-learning; T1 to T50, time point at e-leaning 50 lessons; T51, time point at post completed to e-learning.

Interest in telenursing

This was measured at baseline (T0) and post-completion of the e-learning course (T51) and assessed using a 5-point Likert scale, where 1 = minimum interest and 5 = maximum interest.

E-learning experience

The QUAN learning experience was assessed in terms of usability and practicality. These were assessed for each lesson immediately after its completion using a 5-point Likert scale, where 1 = minimum usability/practicality for me and 5 = maximum usability/practicality for me (T1 to T50).

The course outcomes were considered to be nurses’ confidence in their understanding of telenursing and their satisfaction with the learning experience.

Confidence in their understanding of telenursing was self-reported using a 5-point Likert scale, where 1 = minimum confidence and 5 = maximum confidence before and after the course completion (T0 and T51). Satisfaction with the learning experience was measured using a 0–10 rating on a visual analog scale post-completion (T51).

QUAL

The following data regarding the QUAL learning experience were collected.

Obtained Knowledge and Experience through E-Learning in Each Lesson

We collected open-ended responses from nurses after they completed each lesson (T1 to T50) and explored the knowledge and experience they had acquired (Table 2). All data were collected in Japanese. Irrelevant comments, such as those that did not reference the learning experience, were excluded from the analysis.

Data analyses

QUAN (statistical analysis)

Nurses’ interest and confidence in their understanding of telenursing were analyzed using pre- and post-Wilcoxon matched-pair signed-rank tests. Usability, practicality, and satisfaction were analyzed using an independent samples t-test between the improved (pre–post ≥ 1.0 points) and unimproved (pre–post ≤ 0 points) confidence groups. The significance level was set at p < .05. Statistical analysis was performed using the Japanese version of IBM® SPSS® Statistics (version 27.0).

QUAL

The grounded text mining approach³⁹ was adopted using KH Coder 3.⁴⁰ These analyses examined figure word frequency, co-occurrence network, correspondence of words, and keywords-in-context concordance to obtain a complete picture of the text.^40,41 KH Coder produces a two-dimensional scatter diagram to help visualize the results of the correspondence analysis. This scatter diagram is used to explore the types of words that have a similar appearance pattern⁴⁰ in the improved and unimproved confidence groups. Simultaneously, the first, second, and fourth authors read the open-ended participant responses of both groups and selected descriptions of the knowledge that had been acquired through each lesson using QUAL content analysis techniques. Codes were extracted, and themes were generated and divided into two concepts—usability and practicality in each group. Both codes and themes were discussed among the authors, who have experience in qualitative studies, in the multiple-study project until a consensus was reached. The data were analyzed in Japanese and subsequently translated into English. A back-translation into Japanese confirmed the accuracy of the English translation.

Mixed-methods convergent parallel design data analysis and data integration

After completing the QUAN and QUAL data analyses independently, both types of data were comprehensively integrated using a joint display analysis. A joint display is a graph or table in which researchers juxtapose their QUAN and QUAL results^26,27 to draw meta-inferences—that is, the integrated findings of mixed-methods studies.⁴² Fetters^33(p194) defined joint display analysis as “the process of discovering linkages between the QUAN and QUAL constructs, organizing and recognizing the findings into a matrix or figure to optimize the presentation as a finalized joint display, and interpreting the mixed findings.” A joint display is also useful for presenting QUAN, QUAL, and mixed-methods findings extracted as meta-inferences.^33,43 A deeper understanding of participants’ subjective learning experiences was obtained by examining how they could be linked to improvements (or lack thereof) in their levels of confidence in their understanding of telenursing, and, consequently, their satisfaction with the learning experience. This process allows the drawing of realistic conclusions regarding overall integrative interpretations. Interpretive rigor, which is “the degree to which you make credible interpretations based on the obtained results,” can be established by meeting the key criteria of theoretical consistency and interpretive consistency, agreement, distinctiveness, efficacy, and correspondence^{29(pp320–321)}, as well as inside–outside validity, commensurability, conversion validity, and integration validity.⁴⁴ We attempted to meet these criteria through active discussions between members of an interdisciplinary study team, including nursing, healthcare, evaluation, and mixed-methods researchers.

Ethical considerations

We provided participants with an explanation of the study's purpose and methods and informed them that they could withdraw at any time, without penalty. Enrollment commenced after participants provided online informed consent. This study was approved by the Ethics Committee of St. Luke's International University.

Results

Participants

A total of 227 Japanese nurses were recruited for this study, 152 of whom consented to participate and completed the e-learning program. Among these, nine participants were excluded from the analysis because they were still learning beyond the study end date and, thus, lacked post-course (T51) data. Consequently, the analysis included data obtained from 143 participants (94.1%), 88.8% of whom were women. Their mean age was 44.5 years (SD = 9.5), and all participants were able to operate the course independently. They were divided into two groups based on the scores of their post-course confidence regarding understanding of telenursing, which improved for 104 participants. We called this group the “improved confidence group” as they showed a significant improvement: M = 1.6 (SD = 0.8) points (T0: baseline) and M = 3.4 (SD = 0.8) (T51: after the course) (p < .001). The confidence of 39 participants did not improve or declined. We referred to this group as the “unimproved confidence group” as they showed a significant decline in confidence: M = 3.0 (SD 1.2) points (T0) and M = 2.6 (SD 1.2) (T51) (p = .002). The baseline characteristics of the groups showed no significant differences, except for the mean confidence score. Although the SD was wider in the unimproved group, the mean period of e-learning completion was approximately 20 days for both groups (Table 3).

Table 3.

Characteristics of participants at baseline, n ＝ 143.

	Group
Baseline characteristic	Improved confidence (n = 104)	Unimproved confidence (n = 39)
Age, M(SD), years old	44.0 (9.6)	45.8 (9.3)
Nursing experience, M(SD), years	19.2 (9.8)	20 (10.2)
Full-time worker, n (%)	81 (77.9)	28 (71.8)
Affiliation, n (%)
Clinical	74 (71.2)	27 (69.2)
University/vocational school	12 (11.5)	8 (20.5)
Graduate student	13 (12.5)	3 (7.7)
None	5 (4.8)	1 (2.6)
E-learning period, M(SD), days	17.0 (25.4)	21.5 (43.0)
Access to e-learning, M(SD)
6 am–11:59 a.m.	22.9 (23.0)	17.5 (15.4)
12 pm–17:59 p.m.	40.0 (26.0)	42.2 (26.1)
18 pm–23:59 p.m.	41.3 (28.8)	47.8 (25.3)
0 am–5:59 a.m.	8.3 (15.6)	6.7 (13.7)
Interest score (T0), M(SD)	3.8 (1.1)	4.1 (1.1)
Confidence score (T0), M(SD)	1.6 (0.8)*	3.0 (1.2)*

Participants’ interest in telenursing at baseline

The score for interest in telenursing showed M = 3.8 (SD =1.1) points (T0) among the improved confidence group (n = 104) and M = 4.1 (SD =1.1) (T0) among the unimproved confidence group (n = 39). No statistically significant difference was observed between the two groups (p = .184) (Table 3).

Participants’ e-learning experience: usability and practicality

QUAN

QUAN strands in usability and practicality were as follows.

Usability

The usability score for all 50 lessons showed M = 4.3 (SD = 0.5) points and M = 4.2 (SD = 0.5) in the improved and unimproved confidence groups, respectively. No statistical difference was observed between the two groups (p = .644).

Practicality

The practicality scores for the 50 lessons were M = 4.3 (SD = 0.5) and M = 4.3 (SD = 0.5) in the improved and unimproved confidence groups, respectively. No statistically significant difference was observed between the two groups (p = .880).

QUAL

Eighty-nine (85.6%) participants in the improved confidence group and 33 (84.6%) in the unimproved group provided detailed descriptions of more than one lesson. The number of descriptions per participant that included details of the knowledge acquired was M = 11.9 (SD = 12.4) (range 0–47) and M = 13.8 (SD = 11.8) (range 0–44) in the improved and unimproved confidence groups, respectively. Table 4 lists the 20 most frequent occurrences in each group based on the text mining word frequency analysis. The words “understanding,” “telenursing,” “think,” “necessity,” and “feel” were the five most common; others differed in the order of frequency and word type.

Table 4.

Word frequency.

Group
Improved confidence (n = 89)		Unimproved confidence (n = 33)
Word	Frequency	Word	Frequency
Understanding	394	Understanding	229
Telenursing	391	Telenursing	207
Think	291	Think	147
Necessity	194	Necessity	103
Feel	165	Feel	92
Patient	123	Nursing	75
Lessons	122	Learning	49
To know	114	Information	48
Nursing	109	To be aware of	47
Information	102	Important	43
Value	99	To conduct	41
Specific	94	Value	41
Learning	82	Chapter lessons	40
To be aware of	80	Patients	37
Telenurse	68	User	36
Monitoring	68	Monitoring	35
Actual	67	Telenurse	34
Image	64	Explanation	33
To conduct	64	Support	32
Methods	64	To follow up	32

Figure 3 illustrates the correspondence analysis of words from participants’ responses. The vertical and horizontal axes show the inertia (i.e., variance) of the dimension, which indicates the knowledge acquired through e-learning, and a similar appearance of the word patterns in both groups plotted at the intersection (0.0). The results indicated that both axes represented the same dimension (dimension 1), and the figure was drawn in two dimensions at a 45° angle. The words “needs,” “telenursing,” “information,” “methods,” “think,” “occasion,” “example case,” “pay attention,” and “actual” were similar in both groups. Words, such as “basic,” “learning,” “understand,” and “specific,” were typically extracted for the improved confidence group and “nurse,” “can understand,” “person,” and “important” for the unimproved group. The correspondence analysis showed that the response patterns of the groups were different.

Figure 3.

Correspondence analysis of word frequencies between groups.

Differences in learning experiences

As there were qualitative differences in participant responses between the two groups, we analyzed the descriptions separately for each group by examining their content while raising the level of abstraction in each code. The qualitative themes extracted from these codes were integrated into the concept of usability and included the following:

recognizing the importance of telenursing,

recognizing the value of telenursing,

finding the learning materials uncomplicated, and

critiquing lessons.

The qualitative themes extracted from these codes were integrated into the concept of practicality and included the following:

reflecting on personal communication,

advancing the understanding of and perspective on telenursing,

gaining a general idea of telenursing,

updating knowledge,

understanding patient needs, and

taking on new telenursing responsibilities.

Table 5 presents the joint display of the QUAN and QUAL data about the learning experience as a result of mixed-methods integration to show the differences in the depth of participants’ learning. This converged with the related themes, codes, and participants’ typical descriptions. The usability scores showed no difference (p = .644) across the two groups; however, the participants’ QUAL responses regarding their learning experiences revealed differences in the perceived depth of the learning process between the two groups, except for “recognizing the importance of telenursing.” That is, with themes, such as “recognizing the value of telenursing” and “critiquing lessons,” the improved confidence group showed a deeper understanding of disease monitoring and observation strategies than their counterparts. In contrast, the unimproved group merely indicated that they were aware of telenursing, telenursing methods, and requests involving telenursing. As for the theme of “finding the learning materials uncomplicated,” only the improved confidence group noted that the teaching materials were easy to understand. The unimproved confidence group made no statements evaluating the teaching materials. Under the theme of “critiquing lessons,” the improved group provided constructive opinions, while the unimproved group showed skepticism about telenursing.

Table 5.

Joint display of the e-learning experience.

	Group
	Improved confidence(n = 104)			Unimproved confidence(n = 39)			Meta-inference
Theme	QUANM (SD)	QUALQualitative specific codes	Example description	QUANM(SD)	QUALQualitative specific codes	Example description	Meta-inference
Usability themes	4.3 (0.5)^†			4.2 (0.5)^†			Incongruence. Usability score showed no statistical difference; however, improved confidence group experiences suggested deeper understanding of the usability of the e-learning content, importance and value of telenursing, while the unimproved confidence group only recognized the existence of telenursing or showed some skepticism about it.
Recognizing the importanceof telenursing		Importance of telenursing	"I thought it was important to understand both numerical information and patient's subjective information in remote observation.''		Importance of telenursing	"I recognized the importance of having a perspective to ensure quality."
Recognizing the value of telenursing		Specific understanding of disease monitoring and observation strategies	"I understood the significance, points to keep in mind when telenursing is used for patients with intractable neurological diseases."		Knowing about telenurses, telenursing methods andwhat to do	"I understood about health literacy, communication and telementoring."
Finding the learning materials uncomplicated		Easy to understand	”Tele-monitoring and observation methods are specific but easy to understand.”		None	—
Critiquing lessons		Constructive opinions	"It would be nice to have a little more volume in the test", "It is better to provide information using animation.''		Skeptical of telenursing	"Isn't there some people who are not suitable for telenursing? "
Practicality themes	4.3 (0.5)^††			4.3 (0.5)^††
Reflecting on personal communication		Reflecting own communication	“It was an opportunity to reflect on my communication and actions.”		Reflecting own communication	"It is difficult to communicate in easy-to-understand language, so I would like to try to explain things in my own words on a regular basis."	Incongruence. The practicality score showed no difference; however, the only improved confidence group stated the relevance related to telenursing practice through clearly get a concrete image of telenursing, furthermore, by organizing the basic knowledge and getting a deeper understanding of patient's needs to practice telenursing, the learners found its new role in nursing practice. On the other hand, the unimproved group seemed to have grasped only the rough image of telenursing and could not update their knowledge of telenursing.
Advancing the understanding of and perspective on telenursing		Consideration of matters to be understood in the practice of telenursing	"Telenursing is a new form of nursing, but I think it is necessary to learn and understand what kind of analysis should be done in order to ensure the quality of nursing."		Consideration of matters to be understood in the practice of telenursing	”We felt that there are many things to consider when monitoring areas where users feel shame.”
Gaining a general idea of telenursing		Get a concrete image of telenurse's support	"I was able to visualize how the user measures, how the actual interview is, and what specific support items are."		Grasping the general image of telenursing	"It was a content that gave me a rough image of how to actually run telenursing."
Updating knowledge		Organizing the basic knowledge to practicing telenursing	"It helped me organize my knowledge about telenursing for cancer patients."		Update knowledge in general nursing practice	"Once again, I re-learned about interstitial pneumonia and self-care support."
Understanding patient needs		Got a deeper understanding of patient needs for telenursing	"I have deepened my understanding of the need for telenursing for cancer patients.''　"I thought telenursing was also effective in supporting anxiety, etc."		Understand the needs to conduct telenursing	"I was able to understand the effectiveness of telenursing through case studies."
Taking on new telenursing responsibilities		Found the potential new type of nursing care	"It was a new discovery to be able to practice telenursing to give users a relief of being connected."		None	—

Note. QUAN = quantitative; QUAL = qualitative; M, mean; SD, standard deviation; score range, 1–5.

†t-test p = 0.644. ††t-test p = 0.880.

The practicality scores also showed no statistically significant difference between the two groups. However, qualitative themes, such as “gaining a general idea of telenursing,” “updating knowledge,” “understanding patient needs,” and “taking on new telenursing responsibilities,” which were extracted for the improved confidence group confirmed the differences between the two groups in terms of the participants’ depth of learning and their level of internalizing the knowledge taught, except for “reflecting on personal communication” and “advancing the understanding of and perspective on telenursing.”

E-learning satisfaction

The satisfaction score for the learning experience showed M = 8.4 (SD = 1.2) and M = 8.0 (SD = 1.5) in the improved and unimproved confidence groups, respectively. No statistically significant difference was observed between the two groups (p = .136).

Discussion

In this section, we converge the QUAN and QUAL components to understand how they are linked to participants’ confidence in their understanding of telenursing gained through our course. Moreover, we address the research question by exploring how nurses’ learning experience is linked to their confidence improvement.

Completion of the e-learning course

As most participants (94.1%) completed the e-learning program within the study period, we concluded that their attention to self-learning was sufficient. A combination of stimulating emotions and good instructional content can make e-learning more effective and increase learning motivation.^45,46 We facilitated program completion by including interesting content and case scenarios. The learners’ interest determines their level of participation and the effort needed to learn and affects the learning experience.⁴⁵ Consequently, this e-learning course may have guaranteed the improved confidence group a relevant learning process to obtain knowledge on telenursing, allowing them to self-learn at their own pace while continuing their clinical activities. Moreover, both groups completed the course in approximately 20 days, showing no difference in background and satisfaction with the learning experience.

Nurses’ confidence in their understanding of telenursing

Regarding confidence levels, 72.7% of the participants showed an improvement, whereas the others showed either no change or decreased confidence. Participants’ confidence in their understanding of telenursing at baseline (T0) was higher (M = 3.0 out of 5 points) in the unimproved group and significantly different (p < .001) from the improved confidence group (M = 1.6 points). The correlation between confidence and experience with ICT negatively influenced their attitude toward ICT in health care education.⁴⁷ In this study, a high level of confidence in the unimproved group at baseline indicated a ceiling effect, which may have reflected high expectations from telenursing e-learning and the curiosity to learn. People who are overconfident about telenursing might have a metacognitive bias and overestimate their skills and knowledge, leading to erroneous conclusions⁴⁸ and the further possibility of not paying attention to new information.¹⁵ The role of confidence is to help learners succeed;¹⁵ therefore, having an appropriate level of confidence at baseline may be important for shaping a positive learning attitude and achievement.¹⁵

We found a major qualitative difference between learned experiences in the improved and unimproved confidence groups from the correspondence and content analysis results. In particular, the improved confidence group used more words relating to comprehension and intellect, such as “understand,” “learning,” and “think,” while vocabulary indicating a lower level of comprehension, such as “want to know” and “necessity,” often appeared in responses from the unimproved confidence group. Moreover, only one dimension was extracted in the correspondence analysis; however, the scatter diagram consisted of two axes for convenience owing to data notation and programming.⁴⁹ This indicates that the depth of learning experience was linked to nurses’ confidence improvement, addressing our research question.

We focused on the difference between learned telenursing knowledge and integrated the QUAN and QUAL results. In the QUAL learning process, the results of our content analysis showed the importance of the usability of telenursing e-learning in both groups; however, the codes of the two groups revealed some differences. For example, the theme of “recognizing the value of telenursing” was illustrated by a “specific understanding of disease monitoring and observation strategies” in the improved group and by “knowing about telenurses, telenursing methods, and what to do” in the unimproved group. Skepticism about telenursing was extracted only from the unimproved group, while the improved participants provided constructive opinions on how to refine the lessons. As integrated findings, the usability assessments from the QUAN and QUAL strands showed incongruence.

In practice, the improved confidence group formed a concrete image of disease-specific telemonitoring/telenursing/telenurses, recalled and organized the basic knowledge of telenursing practice, and reached a deeper understanding of patient needs for telenursing. These learning experiences led to recognizing the potential for a new type of nursing care—nevertheless, the unimproved confidence group reported that they were able to gain a general image of telenursing.

However, the richness and the depth of the whole picture of the telenursing learning process created a meta-inference revealing incongruent findings between the QUAN and QUAL data (Table 5). Personal characteristics, such as curiosity (attention) and motives (relevance), determine confidence in the effort and performance of learning.¹⁵ The difference in the depth of the learning process suggests that the unimproved confidence group found the course's usability inadequate and did not perceive the relevance of their learning.¹⁵ To stimulate the motivation to learn, stakeholders should focus more on connecting the relevance of a nurse's specialty and nursing field to the course's learning goals in their revisions of telenursing lessons. External inputs influence and interact with the learner's psychological environment;¹⁵ therefore, the QUAL experience was perceived as a learning attitude based on the premise of the future practice of telenursing and relevance to the learner's specialty.

Although the depth of comprehension observed in the QUAL data seemed to differ across the groups, the usability and practicality scores showed no statistical difference. That is, there was an incongruence between the QUAN and QUAL findings, and the meta-inference showed that the unimproved participants may have rated the usability of the telenursing knowledge gained without an in-depth understanding of the practice. Moreover, motivational aspects, such as the learning system's usability, learner's curiosity, comprehensiveness, and challenges, are essential for achievement.²⁰ Thus, we deepened our understanding of how nurses’ post-program confidence in their understanding of telenursing is linked to their depth of learning and knowledge of this practice—a new finding.

From a lifelong learning perspective, nurses can utilize the obtained knowledge to change nursing practices and client outcomes.⁵⁰ Researchers should further explore the relationship between participants’ clinical nursing backgrounds and basic telenursing literacy. Nurses can enhance their e-learning outcomes through direct interaction and communication with other learners,⁵¹ and their learning success relies on their motivation, attitudes, self-efficacy, and self-regulation.^52,53 Therefore, future research should track login time and activities undertaken during e-learning courses to address these issues.

Contribution to mixed-methods research

Regarding the study's methodological aspects, we employed a mixed-methods convergent parallel design. The mixed-methods design has value for solving problems in complex educational or social contexts.^23,29 We collected QUAN data at two time points (T0 and T51) and QUAL data at 50 time points to integrate all the data on the nurses’ e-learning experiences. The study's strength lies in the complex data integration process that allowed us to gain a comprehensive picture of nurses’ learning process and the factors impacting their confidence. We adopted both deductive and inductive approaches to comprehensively understand what affects the learning process. The deductive approach provided QUAN answers about the learning outcomes, while the inductive approach added QUAL value to the learning process. We focused on the participants’ own appraisals of the learning process and did not include external evaluations. Tashakkori et al.²⁹ suggested that inference in the interpretive rigor of the QUAN and QUAL strands follows relevant findings, distinctiveness, and interpretive agreement. Our meta-inferences are rigorous presentations of the integrated study findings.

Implications

Telenursing e-learning courses are rarely reported in Japan. Lifelong learning environments are vital for the future of nursing education, and expanding telenursing into undergraduate and postgraduate education—in addition to on-the-job training—will enhance nurses’ awareness of and interest in this new field. The telenursing educators, including those teaching e-learning courses, should focus on promoting the learner's positive experience and attitude toward online learning, as well as on offering timely feedback to ensure a correct understanding of the provided information. Effective programs must be designed to ensure quality, usefulness, and convenience.

Limitations

We developed our study's e-learning course for telenursing practice rapidly owing to the COVID-19 pandemic. As De Tornyay⁵⁴ noted, reinforcement of learning and attitude are major conditions for learning nursing, and we lack adequate discussion on these topics. Furthermore, there were no comparisons with prior e-learning experiences. In this study, the participants’ self-evaluations indicated their level of confidence in their understanding of telenursing in e-learning. Future research should assess this variable more objectively.

Conclusions

We examined 143 Japanese nurses’ learning experiences in a newly developed telenursing e-learning program using a single-arm intervention mixed-methods convergent parallel design and integrated the nurses’ experiences related to the QUAN and QUAL learning processes. We linked their self-reported scores of pre- and post-course confidence in their understanding of telenursing with qualitative reflections on their learning experience. Among the participants, 72.7% demonstrated an improved level of confidence in their understanding of telenursing after completing the course. The baseline confidence scores were higher in the unimproved confidence group. Despite no statistical differences in the usability and practical scores between the two groups, the qualitative learning experience showed differences in the depth of the knowledge obtained on telenursing, and this was evident, irrespective of whether or not there was improvement in confidence. This suggests that nursing educators, healthcare policymakers, and other stakeholders should consider that e-learners’ overconfidence in their understanding of telenursing and their comprehension of e-learning materials may result in their failure to develop key competencies, skills, and knowledge. Future research should examine how nurses’ specialties and professional experience affect their confidence in their understanding of telenursing.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241257034 - Supplemental material for Japanese nurses’ confidence in their understanding of telenursing via e-learning: A mixed-methods study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241257034 for Japanese nurses’ confidence in their understanding of telenursing via e-learning: A mixed-methods study by Tomoko Kamei, Aki Kawada, Hisako Kakai, Yuko Yamamoto, Yuki Nakayama, Haruhiko Mitsunaga and Naoki Nishimura in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241257034 - Supplemental material for Japanese nurses’ confidence in their understanding of telenursing via e-learning: A mixed-methods study

Supplemental material, sj-docx-2-dhj-10.1177_20552076241257034 for Japanese nurses’ confidence in their understanding of telenursing via e-learning: A mixed-methods study by Tomoko Kamei, Aki Kawada, Hisako Kakai, Yuko Yamamoto, Yuki Nakayama, Haruhiko Mitsunaga and Naoki Nishimura in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors sincerely thank the participants of this study. We also thank Yasuko Igai (RN, PhD), Ikuo Tofukuji (PhD), Keiko Takahashi (RN, PhD), Tomoyo Harada (RN, MSN), and Takuya Kanamori (RN, MSN) for the development of the e-learning lessons and Mizuho Itabashi (RN, MS) for assistance with the qualitative data analysis.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Japanese Grants-in-Aid for Scientific Research (A) (JSPS KAKENHI) (grant numbers 19H01082 and 23H00453).

Ethical approval

The Ethics Committee of St. Luke's International University approved this study (Approval No. 21-A065, UMIN CTR 45679).

Guarantor:

TK.

Contributorship

TK designed the study. TK, AK, HK, YY, YN, HM, and NN collected the data. TK, AK, YY, HK, and HM analyzed the data. All authors prepared, critically reviewed, and approved the final manuscript.

ORCID iD

Tomoko Kamei

Supplemental material

Supplemental material for this article is available online.

References

Billings

DM.

Conversations in e-learning. Pensacola: Pohl Pub, 2002, pp.3–7.

Organisation for Economic Co-operation and Development. E-learning in tertiary education: where do we stand? Paris: OECD Publishing, 2005, p.11.

Feng

Chang

, et al. Systematic review of effectiveness of situated e-learning on medical and nursing education. Worldviews Evid Based Nurs 2013; 10: 174–183.

Naciri

Radid

Kharbach

, et al. E-learning in health professions education during the COVID-19 pandemic: a systematic review. J Educ Eval Health Prof 2021; 18: 27.

Asimakopoulou

. Telenursing in clinical practice and education. Int J Caring Sci 2020; 13: 781–782.

Kamei

. Telenursing and artificial intelligence for oncology nursing. Asia Pac J Oncol Nurs 2022; 9: 100119.

Kamei

Chapter 6, Example of using information system in nursing. In: Ota

Maeda

(eds) Essentials nursing informatics. Tokyo: Ishiyaku Publishers Inc., 2024, pp.133–145. (in Japanese)

Japan Academy of Home Care. Telenursing guidelines. Tokyo: Shorinsha, 2021, pp.4–39. (in Japanese)

Isidori

Diamanti

Gios

, et al. Digital technologies and the role of health care professionals: scoping review exploring nurses’ skills in the digital era and in the light of the COVID-19 pandemic. JMIR Nurs 2022; 5: e37631.

10.

Rutledge

Hawkins

Bordelon

, et al. Telehealth education: an interprofessional online immersion experience in response to COVID-19. J Nurs Educ 2020; 59: 570–576.

11.

Kamei

Yamamoto

Mitsunaga

, et al. Actual conditions and issues of telenursing education programs for nursing universities and graduate schools in Japan: first report—challenges and needs of telenursing education in the undergraduate program. Japanese J of Telemed Telecare 2021; 16: 110–113. (in Japanese)

12.

Edirippulige

Brooks

Carati

, et al. It’s important, but not important enough: eHealth as a curriculum priority in medical education in Australia. J Telemed Telecare 2018; 24: 697–702.

13.

Gagné

Briggs

LJ.

Principles of instructional design. 2nd ed. New York: Holt, Rinehart, and Winston, 1979, pp.9–11.

14.

Keller

. Development and use of the ARCS model of instructional design. J Instr Dev 1987; 10: 2–10.

15.

Keller

JM.

Motivational design for learning and performance: the ARCS model approach. New York: Springer, 2010, pp.43–74.

16.

Dick

Carey

, et al. The systematic design of instruction. 7th ed. London: Pearson Education, 2009, pp.1–13.

17.

Turi

McMenamin

Wolk

, et al. Primary care provider confidence in addressing opioid use disorder: a concept analysis. Res Nurs Health 2023; 46: 263–273.

18.

White

. Motivation reconsidered: the concept of competence. Psychol Rev 1959; 66: 297–333.

19.

International Organization for Standardization. Ergonomics of human-system interaction—Part 11: usability: definitions and concepts, https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en (2018, accessed 29 December 2023).

20.

Srivastava

Chandra

Lam

. Usability evaluation of e-learning systems. In: Khosrow-Pour

(ed) Encyclopedia of information science and technology. 2nd ed. Hershey: IGI Global, 2009, pp.3897–3903.

21.

Al-Smadi

Abugabah

Smadi

. Evaluation of e-learning experience in the light of the COVID-19 in higher education. Procedia Comput Sci 2022; 201: 383–389.

22.

Stemp

Chosh

Khan

, et al. The role of evaluation methods in health-related e-learning: a rapid review. Online Learning 2022; 26: 369–398.

23.

Mertens

DM.

Research and evaluation in education and psychology: integrating diversity with quantitative, qualitative, and mixed methods. 4th ed. Thousand Oaks: Sage, 2015, pp.304–307.

24.

Sammons

The contribution of mixed methods to recent research on educational effectiveness. In: Tashakkori

Teddlie

(eds.), SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks: Sage, 2010, pp.697–720.

25.

Creswell

. Mapping the developing landscape of mixed methods research. In: Tashakkori

Teddlie

(eds) SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks: Sage, 2010, pp.45–68.

26.

Creswell

JW.

A concise introduction to mixed methods research. Thousand Oaks: Sage, 2015, pp.1–9, 35–37.

27.

Kakai

Introduction to mixed methods. Tokyo: Igaku Shoin, 2015, pp.37–52, 72. (in Japanese)

28.

Plano Clark

Ivankova

How to assess mixed methods research? Considering mixed methods research quality. In: Plano Clark

Ivankova

(eds) Mixed methods research: a guide to the field. Thousand Oaks: Sage, 2016, pp.161–188.

29.

Tashakkori

Johnson

Teddlie

Foundations of mixed methods research integrating quantitative and qualitative approaches in the social and behavioral science. 2nd ed. Thousand Oaks: Sage, 2021, p.8, 84–90, 132, 153, 320–321.

30.

Gibson

. Elaboration, generalization, triangulation, and interpretation: on enhancing the value of mixed method research. Organ Res Methods 2017; 20: 193–223.

31.

Molina-Azorin

. The use and added value of mixed methods in management research. J Mix Methods Res 2011; 5: 7–24.

32.

Hirose

Creswell

. Applying core quality criteria of mixed methods research to an empirical study. J Mix Methods Res 2023; 17: 12–28.

33.

Fetters

MD.

The mixed methods research workbook: activities for designing, implementing, and publishing projects. Thousand Oaks: Sage, 2020, pp.194, 204–206.

34.

Kamei

Yamamoto

Nakayama

, et al. Development of telenursing education program and evaluation by the participants. In: The 19th Annual Conference of Japanese Telemedicine and Telecare Association, Miyagi, Japan, 9–10 October 2015, B-2-3, p.49. (in Japanese)

35.

ABCD-Partners. Edulio [e-learning system platform], https://www.edulio.com/ (2021, accessed 29 December 2023).

36.

Bing-Jonsson

Førsund

Hansen Stålesen

, et al. Lifelong learning in community healthcare: testing competence after learning activities in a blended learning space. Scand J Caring Sci 2023; 37: 1057–1066.

37.

Yamaguchi

Sample size design. Chuo-ku: iHOPE, 2011, pp.49–52. (in Japanese)

38.

Ivankova

NV.

Mixed methods applications in action research: from methods to community action. Thousand Oaks: Sage, 2015, pp.185–190.

39.

Inaba

Kakai

Grounded text mining approach: a synergy between grounded theory and text mining approaches. In: Bryant

Charmaz

(eds), SAGE handbook of current developments in grounded theory. Thousand Oaks: Sage, 2019, pp.332–351.

40.

Higuchi

Quantitative text analysis for social research. Kyoto: Nakanishiya, 2021, pp.1–30. (in Japanese)

41.

Higuchi

. A two-step to quantitative content analysis: KH coder tutorial using Anne of Green Gables (part I). Ritsumeikan Social Sciences Review 2016; 52: 77–91.

42.

Teddlie

Tashakkori

Foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks: Sage, 2009, p.152.

43.

Fetters

Tajima

. Joint displays of integrated data collection in mixed methods research. Int J Qual Methods 2022; 21: 1–13.

44.

Johnson

Fetters

MD.

Introduction to mixed methods

. In: Japan Society for Mixed Methods Research (Eds) Introducing mixed methods research. Tokyo: Tomi Shobo, 2016, pp.5–13. (in Japanese)

45.

Allen

MW.

Motivation in learning. In: Michael Allen’s guide to e-learning. 2nd ed. Hoboken: John Wiley & Sons, 2016, pp.75–84.

46.

Ruggeri

Farrington

Brayne

. Global model for effective use and evaluation of e-learning in health. Telemed J E Health 2013; 19: 312–321.

47.

Bloomfield

Fisher

Davies

, et al. Registered nurses’ attitudes towards e-learning and technology in healthcare: a cross-sectional survey. Nurse Educ Pract 2023; 69: 1–7.

48.

Kruger

Dunning

. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol 1999; 77: 1121–1134.

49.

Higuchi

Frequently asked questions. KH Coder. https://khcoder.net/FAQ.html (n.d., accessed 29 December 2023). (in Japanese)

50.

Rouleau

Gagnon

Côté

, et al. Effects of e-learning in a continuing education context on nursing care: systematic review of systematic qualitative, quantitative, and mixed-studies reviews. J Med Internet Res 2019; 21: e15118.

51.

Okubo

Kamei

Kajii

, et al. A study of factor and causal model structure analyses related to e-learning needs of professional nurses. J of Japan Acad of Nurs Sci 2005; 25: 31–38. (in Japanese)

52.

Kamali

Bagheri-Nesami

. The association between online self-regulated learning and e-learning acceptance among medical sciences students during the COVID-19 pandemic. J Nurs Midwifery Sci 2022; 9: 219–223.

53.

Sögüt

Cangöl

Dolu

İ.

The relationship between eHealth literacy and self-efficacy levels in midwifery students receiving distance education during the COVID-19 pandemic. J Nurs Res 2022; 30: e203.

54.

De Tornyay

Strategies for teaching nursing. Hoboken: John Wiley & Sons, 1971, pp.1–17.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB

0.11 MB