Sage Journals: Discover world-class research

Abstract

Objective:

To examine whether a 2-week or 4-week virtual patient simulation program can improve clinical competence for undergraduate emergency medicine education during COVID-19.

Methods:

This randomized controlled trial was conducted between October and December 2021. Final-year medical students were randomized into a 2-week short exposure group and a 4-week long exposure group on a 1:1 allocation. They were assessed using a virtual case with structured scoring system before and after the virtual patient simulation learning program. An end-of-program questionnaire on learners’ perceptions was also administered.

Results:

A total of 138 students were recruited. After the virtual patient simulation program, the virtual case score increment in the long exposure group was significantly higher than in the short exposure group with a mean score difference of 18/100 (95% confidence interval = 9.8 to 26.3). Questionnaire showed generally positive perceptions toward the use of virtual patient simulation program for emergency medicine teaching.

Conclusion:

Virtual reality learning of 2 and 4 weeks’ duration may be beneficial for improving emergency medicine clinical competence for final-year medical students, especially during periods of limited clinical exposure like the COVID-19 pandemic. A longer 4-week virtual patient simulation program may produce a better learning effect on clinical competence.

Keywords

Education medical undergraduate patient simulation clinical competence COVID-19 high-fidelity simulation training

Introduction

The COVID-19 pandemic had disrupted medical education worldwide. Strict social distancing measures had stopped teachers, students, and patients from meeting physically.^1,2 The pandemic had become a catalyst that has changed how human society approaches teaching and learning, including clinical education of healthcare workers.³ One such change was the observed increase in the use of virtual patient simulation (VPS) for medical education. VPS creates a virtual environment where students could interact with a simulated patient. Students could be tasked to handle a medical condition by history taking, physical examination, investigations, and management. Realistic responses like physical signs, investigation results, and change in vital signs are given after every action or omission.

There is increasing evidence showing the effectiveness of VPS teaching in improving healthcare learners’ knowledge.^4–7 However, there was a paucity of robust evidence for the effectiveness of VPS on clinical competencies like clinical reasoning, decision-making skills, and resource prioritization. A randomized controlled trial (RCT) on 168 medical students found significant improvements in assessment performance, including the speed of recognition of mental status and respiratory status, using virtual reality learning compared to traditional learning.⁸ Another study found higher scores in a key feature examination of a virtual reality program compared with small-group problem-based learning about clinical reasoning in a cohort 112 final-year medical students.⁹ Furthermore, an overall positive attitude on VPS was shown in both medical students^8–13 and nursing students.^14–16

However, the optimal duration of the VPS program for undergraduate emergency medicine education has not been examined in an RCT. The objective of this study was to assess short versus long duration VPS learning program on the acquisition of emergency medicine clinical competency.

Methods

Design, setting, and participants

This was a non-blinded RCT with two parallel groups in 1:1 allocation conducted from October to December 2021. All 220 final-year medical students at the Faculty of Medicine, The Chinese University of Hong Kong were invited to participate in this trial. Due to the strict COVID-19 infection control measures, medical students were not allowed to enter the resuscitation room in the emergency department and face-to-face clinical teaching in the emergency department shop floor was suspended. Regular teaching activities could only be continued in the form of face-to-face tutorials on key emergency medicine topics in non-clinical areas and clinical attachment to low-risk clinical areas, such as the emergency medicine ward. The total teaching contact hour was shortened during this period. To maintain medical students’ exposure to resuscitation of critically ill cases, VPS learning on resuscitation of critical cases was chosen as a solution.

Participants were recruited voluntarily. Invitation email was sent to all 220 final-year medical students. All students who returned a fully signed consent form were included in the study. Students who did not attend the VPS orientation program due to any reason were excluded. The hypothesis was that longer duration VPS program will have a greater improvement in clinical competence from structured assessments.

Interventions

Platform

BodyInteract™ (https://bodyinteract.com/), an online platform that provides a high-fidelity VPS learning, was chosen as the teaching tool in this study. The choice was made based on the accessibility, ease of use, cost, variety and number of cases, and the level of realism provided by the simulation. Participants were tasked to handle emergency medical conditions in a virtual resuscitation room with a simulated patient who gives high-fidelity feedback. After completion of each case, a detailed performance evaluation report is automatically generated to provide feedback to the participants. BodyInteract™ provided 30 different clinical cases on most essential and common emergency syndromes such as acute coronary syndrome, asthma, anaphylaxis, and trauma (see Supplementary Table 1). Participants were free to manage as many cases as they wished during the allocated study period.

Randomization and allocation

Consented participants were randomly allocated into the 2-week short exposure group (SEG) or the 4-week long exposure group (LEG) in a 1:1 ratio. The allocation based on a random number table was generated using Microsoft Excel™.

Study procedure

Introductory videos and a sample case demonstration were given to all participants in a structured face-to-face classroom setting before the study began. Each participant was then given a pre-test which was a simulated virtual case on a trauma patient in the BodyInteract™ system and scored on the performance in managing this case. Participants then had a self-study period (either 2 or 4 weeks) during which they could practice with all the emergency cases in their account, including medical, surgical, and traumatic emergencies. At the end of the self-study period, a tutorial was given by an emergency medicine specialist (A.K.K.L.) using the BodyInteract™ system, and constructive feedback was provided to the participants in small groups. After that, each participant was given a post-test which was the same scenario as the pre-test. An online questionnaire was also provided to all the participants to explore their perception of the VPS program.

Assessments

The pre-test and post-test were a built-in scoring system on the BodyInteract™ platform. The participants were tasked to manage a trauma case within 30 min. Their actions were recorded by the system, based on performance scores on physical examination, medical tests, and treatments, with a full mark of 100 (see Supplementary Table 2). The system gave two scores—one score based on correct actions only (i.e. without penalty) and another score with deductions of incorrect or delayed actions (i.e. with penalty). We used the scoring system with deductions of incorrect or delayed actions as the primary outcome for this trial, as it reflects the importance of timely intervention for managing critical cases.

In addition, to ensure that the performance from the built-in scoring system on the BodyInteract™ platform was consistent with face-to-face examination, 10 participants from each group were randomly selected to undertake an additional objective structured clinical examination (OSCE). All OSCE assessments were performed by two emergency medicine specialists (A.K.K.L. and K.K.C.H.), who assessed participants’ performance in the management of a VPS case of anaphylaxis over 10 min in a standardized manner. The assessment was done using a standardized checklist with a full mark of 25 (see Supplementary Table 3).

Regarding the timing of assessment, both the SEG and LEG group started the learning program at the same time, but the SEG had the assessment earlier due to the shorter exposure period.

Questionnaire

A 14-item questionnaire on the perception of the VPS learning was designed based on the study from De Ponti et al.¹⁰ and was administered to all participants at the end of the program (see Supplementary Figure 1). The answers were given on a 7-point Likert-type scale. It covered the impact of COVID-19: the feasibility, fidelity, content usefulness, content appropriateness, helpfulness of the VPS program, and the perceived confidence and satisfaction of the participants.

Statistical analyses

Intention-to-treat approach was used for analysis. Independent sample t-test was used to compare the changes in pre-test and post-test scores among the SEG and LEG. In addition, Cohen’s d was calculated for the effect size measure. Regarding missing data, students who did not complete the pre-test, their score was assumed to the mean of the group. For those who did not complete the post-test, the post-test score was assumed to be the same as the pre-test score, that is, no improvement. We set the power of the study to be 0.8 and type I error rate (α) to be 0.05. All tests were two-sided.

Descriptive statistics were used for the analysis of the questionnaire results. Cronbach’s alpha was calculated for the internal consistency of the survey. A rating of 5 or more on the Likert-type scale was regarded as favorable. All statistical analyses were done using R 4.2.0 with R statistics programming,¹⁷ and the graphs were generated with the ggplot2 package.¹⁸

Sample size calculations

Sample size calculations were done using G*Power 3.1.9.7.¹⁹ Taking the effect size of the VPS program as 0.48 according to a meta-analysis by Woon et al.,⁷ 140 participants were needed for pre-test and post-test virtual platform scoring. Furthermore, 20 participants were needed for the OSCE assessment based on the results from Borg Sapiano et al.²⁰

Ethical approval and registration

Approval was granted from the local ethics review board. Written consent was obtained from all participants, and the study complies with the Declaration of Helsinki.²¹ The study was registered in the Chinese Clinical Trial Registry (ChiCTR 2100054000) before commencement and conducted according to the CONSORT guideline.²²

Results

Baseline characteristics

The CONSORT flow diagram was shown in Figure 1. Of the 220 final-year medical students invited, 82 students declined and 138 were recruited. They were randomized equally into the SEG (n = 69) and LEG (n = 69). A total of 49% were male, with the mean age of 23. Few (9%) had prior experience with VPS programs. These baseline characteristics were comparable between the SEG and LEG (Table 1). However, even with the longer period allocated to the LEG, the number of completed VPS case episodes was not different from the SEG. The average number of completed case episodes was 9.3 for the SEG and 8.6 for the LEG.

Figure 1.

CONSORT flow diagram.

Table 1.

Baseline participants’ characteristics.

	Overall (n = 138)	SEG (n = 69)	LEG (n = 69)
Male, n (%)	67 (48.6)	33 (47.8)	34 (49.3)
Age, mean ± SD	23 ± 1	23 ± 1	23 ± 1
Previous VPS experience, n (%)	13 (9.4)	7 (10.1)	6 (8.7)

SEG: short exposure group; LEG: long exposure group; SD: standard deviation; VPS: virtual patient simulation.

A total of 24 participants (17.4%) did not perform the virtual case pre-test (12 in SEG and 12 in LEG), and 4 (2.9%) did not perform the virtual case post-test (1 in SEG and 3 in LEG). They were included in the final analysis with imputation using the conservative estimate of no difference after the SEG and LEG interventions.

Pre-test and post-test performance on virtual case score

The mean pre-test scores were similar with 9.1 for SEG and 8.9 for LEG. The mean post-test scores in the LEG of 60.6 were statistically significantly higher than in the SEG of 42.8, with p < 0.01. The mean difference between pre-test and post-test was 51.7 for the LEG and 33.7 for the SEG, with 18.0 more for the LEG (95% confidence interval (CI) = 9.8 to 26.3, p < 0.01). Cohen’s d was calculated as 0.73 between the SEG and the LEG.

OSCE performance

All selected participants completed the pre-test and post-test OSCE (Table 2). Out of a full mark of 25, the pre-test OSCE score were similar with 16.5 for the SEG and 16.3 for the LEG. The post-test OSCE scores were not significantly different with 19.1 for the SEG and 21.7 for the LEG. The pre-test and post-test difference in mean OSCE score between SEG and LEG was not statistically significant (mean difference = 2.8, 95% CI = 2.2 to 7.8, p = 0.23) (Table 3). Cohen’s d was 0.56 between SEG and the LEG.

Table 2.

Virtual case score results of the short and the long exposure groups.

Virtual case score mean ± SD	Overall (n = 138)	SEG (n = 69)	LEG (n = 69)	SEG vs. LEG
Completed VPS episodes	9.0 ± 11.3	9.3 ± 11.8	8.6 ± 10.8	p = 0.73
Pre-test score (with penalty)	9.0 ± 10.1	9.1 ± 9.0	8.9 ± 11.2	p = 0.92
Pre-test score (without penalty)	31.1 ± 14.9	30.9 ± 13.4	31.4 ± 16.4	p = 0.84
Post-test score (with penalty)	51.7 ± 25.3	42.8 ± 21.3	60.6 ± 26.0	p < 0.01
Post-test score (without penalty)	71.8 ± 20.7	66.7 ± 18.5	76.8 ± 21.6	p < 0.01
Score increase (mean difference) (with penalty)	42.7, 95% CI = 38.3 to 47.1, p < 0.01	33.7, 95% CI = 28.4 to 39.0, p < 0.01	51.7, 95% CI = 45.3 to 58.2, p < 0.01	18.0, 95% CI = 9.8 to 26.3, p < 0.01
Score increase (without penalty)	40.6, 95% CI 36.6 to 44.7, p < 0.01	35.8, 95% CI = 30.5 to 41.1, p < 0.01	45.4, 95% CI = 39.4 to 51.4, p < 0.01	9.6, 95% CI = 1.6 to 17.5, p = 0.02

SEG: short exposure group; LEG: long exposure group; SD: standard deviation; VPS: virtual patient simulation; CI: confidence interval.

Table 3.

OSCE results of the short and long exposure group.

In-person OSCE score mean ± SD	Overall (n = 20)	SEG (n = 10)	LEG (n = 10)	SEG vs. LEG
Pre-test	16.4 ± 2.9	16.5 ± 2.6	16.3 ± 3.2	0.9
Post-test	20.4 ± 4.1	19.1 ± 5.2	21.7 ± 2.3	0.2
Score increase (mean difference)	4.0, 95% CI = 1.6 to 6.4, p < 0.01	2.6, 95% CI = −1.2 to 6.4, p = 0.15	5.4, 95% CI = 2.0 to 8.8, p < 0.01	2.8, 95% CI = 2.2 to 7.8, p = 0.23

SEG: short exposure group; LEG: long exposure group; SD: standard deviation; OSCE: objective structured clinical examination; CI: confidence interval.

Participants’ perceptions

The evaluation survey was completed by 71 out of the 138 participants with a response rate of 51.4%. The internal consistency of the survey was found to be high (Cronbach’s alpha = 0.89). A total of 68 (95.8%) participants expressed that the COVID-19 pandemic had greatly affected their learning in clinical medicine. The overall perception of the respondents to the VPS program was positive, with 91.1% responding favorably. Most of the respondents (69/71, 97.2%) agreed that the VPS program should continue after the COVID-19 pandemic ends. All questions and the results were shown in Supplementary Figure 1.

Discussion

To our knowledge, this was the first RCT to examine the effects of a virtual reality learning program using structured virtual and OSCE assessments. The results showed that in both the SEG and LEG, the performance in the virtual case score significantly improved after the VPS program. These results support the notion that virtual reality learning is beneficial for learning clinical competence, echoing the results from previous studies.^23,24

The learning in our VPS program was entirely student-centered, self-directed, and internally motivated. The program was designed to achieve the goal of student-centered learning, which has been shown to be more motivating and effective than traditional learning.^25–27 What was intriguing was that even though both SEG and LEG had a similar number of completed VPS case episodes, the LEG had a significantly higher score increase between the pre- and post-test in the virtual case scores. However, this difference was not seen with the in-person OSCE results, probably due to the small sample size. Future studies should look into how students learn on VPS platforms, and how a longer duration of learning can benefit emergency medicine learning.

It was difficult to explain why a longer permitted self-learning period correlated with better improvements in clinical competency without higher number of completed sessions. It was possible that more time could have allowed more in-depth self-reflection and improvement through self-learning outside the VPS. A longer study period for VPS could also allow a better allocation of time for different learning activities. It was also possible that off-line peer-learning through exchanging information on the virtual learning experience occurred. The ideal duration of virtual learning experience to supplement regular teaching remains unclear in the literature. However, a lack of time was found to be one of the barriers to self-directed learning.²⁸

Study participants expressed serious concerns about the disruption of clinical teaching due to the COVID-19 pandemic, which resonated with other studies.^29,30 Respondents were generally positive about the virtual reality learning experience, praising its ability to deliver an immersive, realistic, and motivating environment. The content was regarded as useful, and participants wanted the VPS program to continue after the pandemic. However, the participants reflected that technical issues were often a problem in delivering virtual reality simulation learning, which was also shown in other studies.^10,14,16 This indicates a need for on-going educator technical support during the VPS sessions and improving student digital literacy in the future medical curriculum.

Limitations

Despite the strength of the RCT design that ensured unmeasured factors such as motivation and intellect could be balanced, there are several limitations to this study. First, the student’s learning process could not be completely controlled by the study intervention. That means, the improvement in clinical competence might be caused by other factors such as self-studying with other means. Second, there was no comparison between VPS and traditional clinical teaching. It was possible that traditional clinical teaching or clinical teaching with restricted exposure to resuscitation cases during the COVID-19 pandemic alone without VPS might still result in similar outcomes. While depriving medical students of any clinical teaching is not feasible, we chose to compare the effect of complementing restricted clinical teaching with VPS on resuscitation cases of different durations and showed that a longer VPS program led to a larger gain in clinical competence. It is noteworthy that this study was conducted in an extraordinary time when traditional clinical teaching was largely unavailable, and students may thus behave differently when COVID-19 is no longer a barrier to traditional teaching and learning. Third, we failed to show a statistically significant difference in the face-to-face OSCE scores possibly due to the small number of participants. Also, nearly half of the participants (48.6%) did not complete the post-program questionnaire, and this might have resulted in responder bias. Therefore, caution is required when interpreting the overwhelmingly positive perception reported by the questionnaire respondents. Finally, this study did not examine some important domains of clinical competence such as psychomotor skills and professionalism. VPS may not be ideal in delivering education in these aspects.

Conclusion

Virtual reality learning of 2 and 4 weeks’ duration may be beneficial for improving emergency medicine clinical competence for final-year medical students, especially during periods of limited clinical exposure like the COVID-19 pandemic. This RCT suggests a longer VPS program with 4 weeks of exposure produced a better learning effect on clinical competence, except for the psychomotor skills and professionalism.

Supplemental Material

sj-docx-1-hkj-10.1177_10249079231189376 – Supplemental material for Virtual patient simulation in undergraduate emergency medicine education during COVID-19: Randomized controlled trial

Supplemental material, sj-docx-1-hkj-10.1177_10249079231189376 for Virtual patient simulation in undergraduate emergency medicine education during COVID-19: Randomized controlled trial by Alex Kwok Keung Law, Kevin Kei Ching Hung, Kirstin Stuart James, Anne-Maree Kelly and Colin A Graham in Hong Kong Journal of Emergency Medicine

Footnotes

Author contributions

A.K.K.L., K.K.C.H., and K.S.J. contributed to the conception of the study. A.K.K.L. and K.K.C.H. coordinated the acquisition of the study data. A.K.K.L. analyzed the data, and A.K.K.L., K.K.C.H., K.S.J., A.-M.K., and C.A.G. interpreted data for the work. A.K.K.L. and K.K.C.H. drafted the manuscript; all authors critically revised the manuscript and agreed to the final version of the manuscript. All authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Availability of data and materials

The anonymised raw data of the study are available from Mendeley Data (, DOI: 10.17632/kz6v7ssnyz.1).

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors declare that they have no competing interests. The investigators received no funding from BodyInteract™. BodyInteract™ and its representatives had no role in this study.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Consent for publication

The manuscript did not contain individuals’ person detailed data in any form.

Ethical approval and consent to participate

The approval was granted from the Joint Chinese University of Hong Kong—New Territories East Cluster Clinical Research Ethics Committee (reference no. 2021.371T). Written consent was obtained from all participants.

ORCID iDs

Kevin Kei Ching Hung

Colin A Graham

Supplemental material

Supplemental material for this article is available online.

References

Akers

Blough

Iyer

MS.

COVID-19 implications on clinical clerkships and the residency application process for medical students. Cureus 2020; 12(4): e7800.

Ferrel

Ryan

JJ.

The impact of COVID-19 on medical education. Cureus 2020; 12(3): 10–13.

Gibbs

The Covid-19 pandemic: provoking thought and encouraging change. Med Teach 2020; 42(7): 738–740.

Lebdai

Mauget

Cousseau

, et al. Improving academic performance in medical students using immersive virtual patient simulation: a randomized controlled trial. J Surg Educ 2021; 78(2): 478–484.

Mansoory

Khazaei

Azizi

, et al. Comparison of the effectiveness of lecture instruction and virtual reality-based serious gaming instruction on the medical students’ learning outcome about approach to coma. BMC Med Educ 2021; 21(1): 1–7.

Watari

Tokuda

Owada

, et al. The utility of virtual patient simulations for clinical reasoning education. Int J Environ Res Public Health 2020; 17(15): 1–9.

Woon

APN

Mok

Chieng

YJS

, et al. Effectiveness of virtual reality training in improving knowledge among nursing students: a systematic review, meta-analysis and meta-regression. Nurse Educ Today 2021; 98: 104655.

Yang

Buck

Evans

, et al. A telesimulation elective to provide medical students with pediatric patient care experiences during the COVID pandemic. Pediatr Emerg Care 2021; 37(2): 119–122.

Perron

Coffey

Lovell-Simons

, et al. Resuscitating cardiopulmonary resuscitation training in a virtual reality: prospective interventional study. J Med Internet Res 2021; 23(7): 1–10.

10.

De Ponti

Marazzato

Maresca

, et al. Pre-graduation medical training including virtual reality during COVID-19 pandemic: a report on students’ perception. BMC Med Educ 2020; 20(1): 332.

11.

Taubert

Webber

Hamilton

, et al. Virtual reality videos used in undergraduate palliative and oncology medical teaching: results of a pilot study. BMJ Support Palliat Care 2019; 9(3): 281–285.

12.

Hardcastle

Wood

The utility of virtual reality surgical simulation in the undergraduate otorhinolaryngology curriculum. J Laryngol Otol 2018; 132(12): 1072–1076.

13.

Mills

Dykstra

Hansen

, et al. Virtual reality triage training can provide comparable simulation efficacy for paramedicine students compared to live simulation-based scenarios. Prehosp Emerg Care 2020; 24(4): 525–536.

14.

Saab

Hegarty

Murphy

, et al. Incorporating virtual reality in nurse education: a qualitative study of nursing students’ perspectives. Nurse Educ Today 2021; 105: 105045.

15.

Berman

Biguet

Stathakarou

, et al. Virtual patients in a behavioral medicine massive open online course (MOOC): a qualitative and quantitative analysis of participants’ perceptions. Acad Psychiatry 2017; 41(5): 631–641.

16.

Kidd

Knisley

Morgan

KI.

Effectiveness of a second life^® simulation as a teaching strategy for undergraduate mental health nursing students. J Psychosoc Nurs Ment Health Serv 2012; 50(7): 28–37.

17.

R Core Team. R: a language and environment for statistical computing, 2020, https://www.r-project.org/ (accessed 25 May 2023).

18.

Wickham

ggplot2: elegant graphics for data analysis [Internet]. New York: Springer, 2016, https://ggplot2.tidyverse.org (accessed 25 May 2023).

19.

Faul

Erdfelder

Lang

, et al. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007; 39(2): 175–191.

20.

Borg Sapiano

Sammut

Trapani

. The effectiveness of virtual simulation in improving student nurses’ knowledge and performance during patient deterioration: a pre and post test design. Nurse Educ Today 2018; 62: 128–133.

21.

World Medical Association. WMA declaration of Helsinki—Ethical principles for medical research involving human subjects [Internet], https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/ (accessed 25 May 2023).

22.

Schulz

Altman

Moher

, et al. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Trials 2010; 11(1): 1–8.

23.

Zackoff

Real

Sahay

, et al. Impact of an immersive virtual reality curriculum on medical students’ clinical assessment of infants with respiratory distress. Pediatr Crit Care Med 2020; 21(5): 477–485.

24.

Middeke

Anders

Schuelper

, et al. Training of clinical reasoning with a Serious Game versus small-group problem-based learning: a prospective study. PLoS ONE 2018; 13(9): e0203851.

25.

Wilson

Brown

Misch

, et al. Breaking with tradition: a scoping meta-analysis analyzing the effects of student-centered learning and computer-aided instruction on student performance in anatomy. Anat Sci Educ 2019; 12(1): 61–73.

26.

Cheang

KI.

Effect of learner-centered teaching on motivation and learning strategies in a third-year pharmacotherapy course. Am J Pharm Educ 2009; 73(3): 1–42.

27.

Murad

Coto-Yglesias

Varkey

, et al. The effectiveness of self-directed learning in health professions education: a systematic review. Med Educ 2010; 44(11): 1057–1068.

28.

Jeong

Presseau

ElChamaa

, et al. Barriers and facilitators to self-directed learning in continuing professional development for physicians in Canada: a scoping review. Acad Med 2018; 93(8): 1245–1254.

29.

Choi

Jegatheeswaran

Minocha

, et al. The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey. BMC Med Educ 2020; 20(1): 206.

30.

Nishimura

Ochi

Tokumasu

, et al. Impact of the COVID-19 pandemic on the psychological distress of medical students in Japan: cross-sectional survey study. J Med Internet Res 2021; 23(2): e25232.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.56 MB