Sage Journals: Discover world-class research

Abstract

Background:

There is a plethora of technologies used in the rapidly expanding field of telemedicine, with tele-intensive care units (TICUs) being a specialty with rather distinct needs and challenges. Technology is a crucial aspect of TICU collaboration to ensure high-quality telemedicine consultations in clinical routines.

Methods:

We conducted a feedback survey to assess the technical performance within our international TICU network. The survey, addressing medical staff that regularly participates in telemedicine rounds, was designed to specifically assess the technical performance of a TICU system. In detail, usability, perception of audio–visual quality, and user acceptance were evaluated.

Results:

The results show a positive evaluation by the vast majority of participants, with only minor differences between countries. Potential for improvement was identified in multiple domains closely linked to contextual factors.

Discussion:

The perception of telemedicine technology performance was promising and will guide further research to enhance technological acceptance and optimize TICU networks for widespread use.

Keywords

feedback intensive care unit medical staff technology telemedicine

Introduction

The recent COVID-19 pandemic accelerated telemedicine development globally in many medical specialties and health care settings,¹ including intensive care medicine, to manage complex critically ill patients in a collaborative approach in tele-intensive care unit (TICU) networks.² TICUs have shown to be beneficial to reduce mortality, hospital/intensive care unit (ICU) length of stay, complications, and costs.^3–8 Technologies in telemedicine, in general, vary widely,⁹ ranging, for example, from web-based messaging systems with store-and-forward communication¹⁰ to full-scale real-time monitoring from off-site telemedicine hubs.¹¹ However, telemedicine in intensive care may be technically challenging due to unstable internet connections or interference with a multitude of technical (medical) applications in ICUs, environmental factors (e.g., high noise levels),¹² or the need for a rapid response and decision making related to time-critical changes in patients’ medical condition. Whereas the technology in TICUs used to be rather uniform for many years,¹³ the COVID-19 pandemic led to the widespread use of different devices in critical care, with secure video consultation or meeting software being the predominant technology.^2,14,15 Satisfaction with diverse TICU technologies varies: Nelson et al. reported that up to 66.7% of their users experienced technical issues with telehealth technologies,² whereas Munusamy et al. reported that all staff in their study rated the technical quality as good or excellent,¹⁶ and van der Voort et al. described that only half of their survey respondents rated the audio–visual quality of the technical device in use as good.¹⁷ In congruence with the majority of TICU networks,¹⁸ our collaboration is rooted in a regional TICU network.¹⁹ TICU networks crossing country borders to collaborate across multiple countries are still less frequent.^20–22 Our research-driven national collaboration expanded across country borders during the pandemic,²³ evolving into a regular cooperation, attempting to discuss and treat critical care patients jointly in daily routines. Hitherto the transformation from a homogenous national collaboration to an international network with a multitude of technical standards and prerequisites of diverse health care systems has not been evaluated systematically. Therefore, we initiated a feedback study to analyze TICU service quality, with the technical performance being one of the most crucial domains in everyday use. We aim to elucidate the technical performance, namely usability, audio–visual quality, user acceptance, and potential for improvement, as well as differences between multiple countries against the background of very diverse health care settings with, for example, possible culture and language barriers.

Methods

Study design

We initiated a feedback survey to assess the technical performance of an international TICU network targeting medical personnel, who regularly collaborate in this network. The TICU hub, located at the Department of Anesthesiology and Intensive Care Medicine at a tertiary university hospital in Germany, was—at the time of the study—connected to 10 (peri-) urban hospitals in four lower middle- to high-income countries²⁴ in Europe, Africa, and Asia, with five facilities being located in a conflict-affected situation, as defined by the World Bank Group.²⁵ Two of the participating hospitals are classified as medium-sized (100–499 beds) and eight as large (500 or more beds), with the total number of hospital beds ranging from 355 to 1899. The number of ICUs per facility varied between 1 and >5, whereas the type of ICUs included general, surgical and traumatological, cardiological, oncological, and pediatric ICUs. TICU rounds were scheduled one to five times per week and realized additionally 24/7 on demand, depending on the spoke sites’ needs. The survey was conducted between October 17 and December 6, 2023. This study was approved by the local institutional review board (No. EA2/163/23) and registered on August 11, 2023, with the German Clinical Trials Register (DRKS-IDDRKS00032246).

The survey was carried out as an online questionnaire in English language using the statistical survey web app LimeSurvey (LimeSurvey GmbH, Hamburg, Germany). Medical personnel (i.e., physicians and nurses) in partner hospitals were invited via emails to participate. No identifiable data, including demographic characteristics of the respondents, were acquired, and all participants’ consent was obtained by actively agreeing on study and data protection information before accessing the questionnaire. To explore the technical domain, the survey included two core sections: (i) the System Usability Scale (SUS)²⁶ to analyze usability and allow comparability with other technologies and previous research in a panoply of telemedicine settings^27–30 and (ii) a set of questions to address audio–visual quality and user acceptance according to our TICU-Feedback-Tool proposed previously (Supplementary Table S1).³¹ The TICU-Feedback-Tool was developed in a consensus process and explicitly tested for psychometric properties before starting routine use.³¹ The SUS is a reliable and validated 10-item instrument to evaluate usability on a 5-point Likert scale (ranging from “strongly disagree” to “strongly agree”).²⁶ All, but one, of the questions of the second core section were also rated on a 5-point Likert scale ranging from “strongly disagree” to “strongly agree.” The final question was the single question of the survey asking the participants for a free-text answer regarding potential areas of improvement. Figure 1 displays the course of the study.

FIG. 1.

Flowchart course of study.

TICU technology

The technology in use within the TICU network consists of a mobile device (Fig. 2: mobile device, Lite 3, Teladoc Health^TM; New York, USA) at the patients’ bedside intended to provide high-quality Health Insurance Portability and Accountability Act (HIPAA)-compliant audio and video sessions with the off-site TICU hub (Fig. 3: TICU Hub). The device is equipped with dual high-definition cameras to capture remote video for viewing on the Provider Access Software by the TICU intensivist and a directional microphone to pick up audio in a narrow zone. The touch-sensitive display enables controls by operating medical personnel at the patients’ bedside and shows the TICU hub user’s face. The cameras can be navigated remotely (pan range +/− 170°, tilt range +27°/− 65°) by the TICU specialist via the Provider Access Software³² (Teladoc Health^TM). Connection to the internet is established via conventional Wi-Fi networks or a mobile hotspot, depending on local availability; power for the device is provided by battery when not plugged in. The device is designed for use by trained health care professionals. In addition, 24/7 technical support for hospital teams is granted by the provider (i.e., Teladoc Health).

FIG. 2.

Mobile device and intensivist at a patient’s bedside.

FIG. 3.

TICU Hub: TICU intensivist and Provider Access Software.

Statistical analysis

Data were analyzed with SPSS Statistics 27 (SPSS Inc., Chicago, USA) and R (Version 4.2.1, The R Foundation, Vienna, Austria). Statistical analysis was conducted predominantly using methods of descriptive statistics. The SUS was analyzed by single items (Table 1), and the SUS score was calculated as a composite measure of the overall usability.²⁶ The mean SUS scores were chosen for comparability with prior research. Analysis of variance was used to test for differences in SUS scores between countries. The SUS score was rated according to Bangor’s adjective scale, ranging from “best imaginable” (SUS = 84.1–100) to “worst imaginable” (SUS = 0–25; best imaginable/excellent/good/ok/poor/worst imaginable).³³

Table 1.

Results of the System Usability Scale

Question	Strongly disagree n (%)	Disagree n (%)	Neutral n (%)	Agree n (%)	Strongly agree n (%)	Overall
I think that I would like to use this system/telemedicine device frequently	2 (5.7)		1 (2.9)	15 (42.9)	17 (48.6)	35 (100%)
I found the system/telemedicine device unnecessarily complex	15 (42.9)	16 (45.7)	1 (2.9)	3 (8.6)		35 (100%)
I thought that the system/telemedicine device was easy to use	1 (2.9)	1 (2.9)	2 (5.7)	13 (37.1)	18 (51.4)	35 (100%)
I think that I would need the support of a technical person to be able to use this system	9 (25.7)	13 (37.1)	6 (17.1)	6 (17.1)	1 (2.9)	35 (100%)
I found that the various functions in this system/telemedicine device were well-integrated		3 (8.6)	2 (5.7)	18 (51.4)	12 (34.3)	35 (100%)
I thought that there was too much inconsistency in this system	5 (14.3)	23 (65.7)	4 (11.4)	2 (5.7)	1 (2.9)	35 (100%)
I would imagine that most people would learn to use this system/the telemedicine device very quickly		1 (2.9)	1 (2.9)	21 (60)	12 (34.3)	35 (100%)
I found the system very cumbersome to use	8 (22.9)	12 (34.3)	6 (17.1)	8 (22.9)	1 (2.9)	35 (100%)
I felt very confident using the system/telemedicine device	1 (2.9)		3 (8.6)	18 (51.4)	13 (37.1)	35 (100%)
I needed to learn a lot of things before I could get going with this system/telemedicine device	11 (31.4)	15 (42.9)	7 (20)	2 (5.7)		35 (100%)

Given the limited sample size, nonparametric methods for factorial designs were utilized to compare the TICU Feedback-Tool items between countries. Relative effect estimates (REs) with 95% confidence intervals (CIs) and corresponding two-sided p-values were calculated in R, employing the rankFD package (Version 0.1.1). REs are bounded within the interval [0,1]. Larger REs suggest stronger approval for aspects related to audio–visual quality and user acceptance when comparing across countries.

The p-values below a significance level of ≤0.05 were considered significant. However, due to the exploratory nature of this study, no adjustment of p-values for multiplicity was applied. Hence, p-values are given as an orientation and are not interpreted as confirmatory.

Results

A total of 35 staff members, namely ICU physicians and nurses, from ten hospitals in four countries (Germany: two hospitals, South Africa: two hospitals, Ukraine: five hospitals, Uzbekistan: one hospital) participated, out of which 30 fully completed the survey, whereas 5 solely answered the usability questionnaire (i.e., the SUS). As the survey was sent to one coordinator per spoke site and further distributed within the facility’s ICU team, the total number of individuals receiving the questionnaire remains unknown; therefore, a response rate cannot be provided. Table 1 presents the results of the 35 respondents of the SUS. All questions are displayed with the respective answers, ranging from “strongly disagree” to “strongly agree.”

The mean SUS score was 76 ± 13.7, ranging from 47.5 to 100; therefore, the overall usability of the telemedicine device was rated as “good.” The mean SUS scores, analyzed by countries, showed that the highest mean SUS score was obtained in Germany (84.7 ± 8.4), followed by Uzbekistan (76.4 ± 15.9), Ukraine (75.4 ± 15.0), and South Africa (65.4 ± 7.3), with a significant difference between countries (p = 0.040). Hence, participants in all countries, but South Africa, rated the usability of the technology in use as “good,” whereas South African participants rated the usability of the device as “OK” according to Bangor’s adjective scale.³³

Sound and picture quality were assessed as good and error-free by the majority of participants, with >85% (26 and 28 of n = 30) of participants agreeing or strongly agreeing, as displayed in Figure 4.

FIG. 4.

Sound and picture quality.

Of the bedside medical staff, 83.4% (25 of n = 30) agreed or strongly agreed that the communication with the TICU intensivist was technically functioning well; none of the participants disagreed or strongly disagreed. The attitude of the operating medical personnel in partner hospitals toward telemedicine consultations using this particular technology was generally positive, with 30 (100%) agreeing or strongly agreeing (50%/50%).

For the picture quality, REs showed the highest tendency for stronger approval in German participants (RE = 0.6750, 95% CI [0.5281, 0.7940]), followed by Ukraine (RE = 0.4500, 95% CI [0.3111, 0.5972]), and Uzbekistan as well as South Africa (RE = 0.4375, 95% CI [0.3698, 0.5076]). Sound quality was rated with the highest tendency for stronger approval by medical staff from Uzbekistan (RE = 0.6110, 95% CI [0.4340, 0.7629]), followed by Germany (RE = 0.4978, 95% CI [0.3428, 0.6532]), Ukraine (RE = 0.4475, 95% CI [0.3063, 0.5977]), and South Africa (RE = 0.4438, 95% CI [0.2997, 0.5980]). The tendency for stronger approval regarding the item “From the technical side, communication with the telemedicine physician functioned well” was the highest in German medical staff (RE = 0.6511, 95% CI [0.5010, 0.7762]), followed by Uzbekistan, South Africa, and Ukraine. For the positive attitude toward telemedicine consultations using the device, German participants showed the highest tendency for stronger agreement (RE = 0.5683, 95% CI [0.4062, 0.7170]) compared with Ukraine (RE = 0.5058, 95% CI [0.3579, 0.6527]), South Africa (RE = 0.4701, 95% CI [0.3085, 0.6382]), and Uzbekistan (RE = 0.4558 95% CI [0.2751, 0.6489]).

In total, there was no significant difference in all four items comparing each country with the entire group of participants. Details on the results are displayed in Supplementary Tables S2 and S3.

Of the users, 73.3% (22 of n = 30) agreed or strongly agreed that there was still potential for quality improvement of the telemedicine consultations in general, with the respective question not specifically targeting the technical aspects. In the final open-ended question, one participant expressed the need for better internet connectivity in their ICU, and one participant desired technical improvements, namely more easy-to-use mobile devices.

Discussion

There is a wide range of technologies used in the rapidly expanding field of telemedicine, with tele-critical care just being one of many, but a specialty with rather distinct needs and challenges. The recent COVID-19 pandemic further revealed the shortage of medical staff with intensive care expertise^34,35 and therefore accelerated TICU activities in the global-health context, with an increasing variety of technologies in use.

To ensure sufficient and well-accepted technical standards, we systematically evaluated the performance of the technology used in our TICU network according to the World Health Organization’s (WHO) recommendation.³⁶ This observational study, which uses a survey methodology across the international TICU network, highlights that the perception of technological performance is influenced not only by the technology itself but also by the varying contexts.

The mean SUS score of 76 indicates a “good” usability of the telemedicine device in general, consistent with or even superior to other technologies reported in prior research, ranging from 58.8²⁹ to 70.7³⁰ in diverse telemedicine settings.^27–30 There was a significant difference though between countries, with participants from Germany, Ukraine, and Uzbekistan rating the usability as “good” and South African medical personnel as “OK.”

The perception of audio–visual quality and user acceptance were evaluated positively, with the vast majority of medical staff agreeing or strongly agreeing and only a few participants rating the items as neutral. Congruent with the SUS score results, there was a tendency among South African personnel for lower approval in the sound and picture quality items.

The reason for the variation between hospitals from different countries is not yet fully understood. The duration of collaboration within the TICU network, thus the time to familiarize with the specific technology, might be a reason. Moreover, a language barrier restricting communication or even necessitating translation to approach the telemedicine hub or technical support team as described previously by WHO as one of the main obstacles,³⁷ a heterogeneity in access to diverse technologies and subjective comfort to use new technologies might cause higher or lower acceptance and ease to use the device in daily routines. Furthermore, usability and technical performance might also differ due to the quality of internet connection and power supply, therefore not being specific to the device itself but rather the local context. TICU networks collaborating in very diverse health care systems in multiple countries with intrinsic cultural and linguistic differences as well as additional varying contextual factors such as economic background or conflict-affected situations (e.g., prone to fluctuating availability of power and Wi-Fi or restricted access to/loss of telemedicine devices) are therefore challenging. Hence, further research is needed to elucidate and approach the specific difficulties with the technology in use in diverse settings.

The majority of participants identified potential for improvement. This item did not specifically target the technical performance, but stable internet connections were mentioned to be crucial and more easy-to-use mobile devices desirable for future progress. Satellite-based internet or strong mobile networks might be a solution to reduce unstable Wi-Fi connections. When establishing TICU networks, the focus should be set on the thorough assessment and implementation of these features, as indicated previously by WHO.³⁸

Our study has limitations that should be considered when interpreting the findings. First, we used a survey-based approach, which comes with inherent methodological limitations. For example, only English-speaking staff were able to participate, which may have excluded potential respondents in non-native speaking contexts. Additionally, the recall rate, as commonly seen in online surveys,^39,40 may be limited due to the method of distribution and is unknown because the local staff shared the participation link within their teams. However, we mitigated this by utilizing an established email network and received responses from all participating facilities, which provides a comprehensive view of the network’s performance. Second, the survey was only administered to the remote (bedside-) medical staff and not to the hub personnel, which may limit the overall scope of perspectives on the technology. Third, the small sample size yields a high statistical uncertainty and results in an exploratory character of the study. Fourth, we only evaluated one type of TICU technology, preventing a comparison with other devices or applications, which limits the generalizability of our results to other telemedicine networks. Lastly, the survey was conducted after a crisis period, namely the COVID-19 pandemic, which may not accurately reflect perceptions in a more stable, routine clinical environment.

Despite these limitations, our study also has several strengths. First, our TICU network is technically homogenous, with all participating countries using the same technology, providing a unique opportunity for direct comparisons across different regions and reducing the barriers that are inherent to diverse technologies (e.g., a lack of interconnection).³⁸ This consistency allows us to assess intergeographical differences in technological acceptance more reliably. Second, we used standardized questions, making it possible to repeat the survey in the future and collect longitudinal data for further evaluation. Third, by ensuring strict anonymity for survey respondents, we reduced the risk of social desirability bias. Finally, participation from all countries within the network allowed us to capture a broad range of perspectives and identify regional differences in the perception of telemedicine technology.

Conclusion

In summary, our survey revealed that the perception of telemedicine technology performance is influenced not only by the technology itself but also substantially by the varying contextual factors inherent to TICU collaborations crossing borders. Prospectively, TICU programs may focus on improving internet connectivity and on addressing local factors that hinder the mobility and usability of TICU devices. Nevertheless, further research involving international networks is needed to validate our findings, enhance technological acceptance, and optimize future TICU networks for widespread use in diverse health care systems.

Authors’ Contributions

F.L., K.S., A.H., S.S., and B.W. designed the study. V.T., I.L., K.A., F.P., J.P.K., and J.G. supervised the study at spokes sites. D.H. supported the study management. F.L. and S.S. analyzed the data. F.L., S.S., and B.W. interpreted the data. F.L. wrote the first draft of the paper. S.S., B.W., A.E., and C.D.S. reviewed the article. All authors contributed to finalizing the article.

Footnotes

Acknowledgments

The authors thank the nursing and medical staff in all participating intensive care units for their collaboration.

Author Disclosure Statement

C.D.S. received support for the present article from BMBF/RKI, that is, TICO-COVID-19; FUTURE-MOCCA; FUTURE UGA; FUTURE UZB; FUTURE ZAF; FUTURE UKR, FUTURE WASP (2020–2025), grants or contracts from Deutsche Forschungsgemeinschaft/German Research Society, Deutsches Zentrum für Luft-und Raumfahrt e.V. (DLR)/German Aerospace Center, Einstein Stiftung Berlin/Einstein Foundation Berlin, Gemeinsamer Bundesausschuss/Federal Joint Committee (G-BA), Inneruniversitäre Forschungsförderung/Inner University Grants, Projektträger im DLR/Project Management Agency, Stifterverband/Non-Profit Society Promoting Science and Education, European Society of Anaesthesiology and Intensive Care (public/reviewed grant), BMWI-Federal Ministry for Economic Affairs and Climate Action (ZIM, KF2027803KJ2), Georg Thieme Verlag (Payment for book contribution as author), Dr. F. Köhler Chemie GmbH (IIT; CESARO Study, Phydelio Study, LoveMi Study), Sintetica GmbH (IIT: PODSPA Study), Gemeinsamer Bundesausschuss/Federal Joint Committee (G-BA)-Innovationsfond (2017–2020 ERIC [01NVF16011]), Gemeinsamer Bundesausschuss/Federal Joint Committee (G-BA)-Innovationsfond (2019–2025 PRÄP-GO [01NVF18024]), Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (ERONA 2019–2021), Stifterverband für die deutsche Wissenschaft e.V./Metronic (E-Health shared decision allocation [2018–2024]), Philips ElectronicsNederland BV (2019–2023: ICU Feel Better app), BMBF-Federal Ministry of Education and Research (2020: CEO-sys; EViPan Unimed; CODEX+; Prepared), Deutsche Forschungsgemeinschaft/German Research Society (2020–2026: “Heisenberg professorship”), Gemeinsamer Bundesausschuss/Federal Joint Committee (G-BA)-Innovationsfond (2020–2023 SEPWISS), BMBF (2021–2022 STAIRS; STAIRS RHISSA 2023–2028), Gemeinsamer Bundesausschuss/Federal Joint Committee (G-BA)-Innovationsfond (2022–2024 iWILL, 2022–2024 E = MC² [01VSF21035]), 2023–2025 DigiPOD (01VSF22040)], The European Commission Horizont Europa (2024–2028 EPIC [101137221]), holds patents (planned/issued/pending) Issued EUROPE (GER;AT;CH;LI;DE;FR;GB;NL) Inventor (15753 627.7), Issued United States Patent; Inventor (PCT/EP 2015/067731), Issued EUROPE (GER;CH;LI;DE;FR;NL) Inventor (3 174 588), International Patent Germany (10 2014 215 211.9, 10 2018 114 364.8, 10 2018 110 275.5, 50 2015 010 534.8, 50 2015 010 347.7, 10 2014 215 212.7), participated on a Data Safety Monitoring Board or Advisory Board for Prothor (2023), Takeda Pharmaceutical Company Limited (2024), Lynx Health Science GmbH (2024), holds a leadership or fiduciary role in other board, society, committee, or advocacy group (unpaid) at Association of the Scientific Medical Societies in Germany (AWMF), Deutsche Forschungsgemeinschaft (German Research Foundation) review boards, Deutsche Akademie der Naturforscher Leopoldina e.V.—German National Academy of Sciences—Leopoldina, Berliner Medizinische Gesellschaft, European Society of Intensive Care Medicine (ESICM), European Society of Anaesthesiology and Intensive Care (ESAIC), Deutsche Gesellschaft für Anästhesiologie und Intensivmedizin-German Society of Anaesthesiology and Intensive Care Medicine (DGAI), Deutsche Interdisziplinäre Vereinigung für Intensiv-und Notfallmedizin-German Interdisciplinary Association for Intensive Care and Emergency Medicine (DIVI), Deutsche Sepsis-Stiftung—German Sepsis Foundation. B.W. received grants or contracts from Future-UKR, German Ministry of Health, Innovation Funds 01VSF21035 (2022), payment or honoraria for lectures, presentations, speakers bureaus, article writing, or educational events from Sedana Medical, institutional (2024), Dr. F. Koehler Chemie, institutional and personal (2025), Teladoc Health GmbH (2025), support for attending meetings and/or travel from Teladoc Health GmbH, holds a leadership or fiduciary role in other board, society, committee, or advocacy group (unpaid) as DGAI State Chair Berlin. A.E. received payment or honoraria for lectures, presentations, speakers bureaus, article from Dräger Medical Deutschland GmbH and is a member of the German Society for Anesthesiology and Intensive Care Medicine (DGAI), DGAI Round Table Tele-ICU. F.L., K.S., S.S., A.H., V.T., I.L., K.A., F.P., J.-P.K., J.G., and D.H. declare that they have no competing interests.

Funding Information

This research received public funding from the German Federal Ministry of Health via The Global Health Protection Program (GHPP) as a part of the project: “Strengthening clinical management of COVID-19 patients using telemedicine and web-based learning”–FUTURE International UZB/URK 1–2.” (01VSF21035).

Ethics Approval

This study was approved by the Local Ethics Committee of Charité–Universitätsmedizin Berlin (registered under No. EA2/163/23).

Consent to Participate

Informed consent to participate was obtained from all of the participants in this study prior to participation. The study was conducted in compliance with the Declaration of Helsinki.

Clinical Trial Registry

The study was registered on August 11, 2023, with the German Clinical Trials Register (DRKS-IDDRKS00032246).

Consent for Publication

All people included in Figures 1 and gave written informed consent for their images to be published in this study.

Availability of Data and Materials

The datasets used and/or analyzed during the current study, surpassing the complete version of the survey’s questions, are available from the corresponding author on reasonable request.

Supplemental Material

Abbreviations

References

Alashek

, Ali

. Satisfaction with telemedicine use during COVID-19 pandemic in the UK: A systematic review. Libyan J Med, 2024; 19(1):2301829; doi: 10.1080/19932820.2024.2301829

Nelson

, Steuernagle

, Rotello

, et al. COVID-19 and telehealth in the intensive care unit setting: A survey. BMC Health Serv Res, 2022; 22(1):797; doi: 10.1186/s12913-022-08197-7

Boyle

, Palmer

, Konzen

, et al. Telemedicine critical care-mediated mortality reductions in lower-performing patient diagnosis groups: A prospective, before and after study. Crit Care Explor, 2023; 5(10):e0979; doi: 10.1097/CCE.0000000000000979

Breslow

, Rosenfeld

, Doerfler

, et al. Effect of a multiple-site intensive care unit telemedicine program on clinical and economic outcomes: An alternative paradigm for intensivist staffing. Crit Care Med, 2004; 32(1):31–38; doi: 10.1097/01.CCM.0000104204.61296.41

Lilly

, Cody

, Zhao

, et al.; University of Massachusetts Memorial Critical Care Operations Group. Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. JAMA, 2011; 305(21):2175–2183; doi: 10.1001/jama.2011.697

Lilly

, McLaughlin

, Zhao

, et al.; UMass Memorial Critical Care Operations Group. A multicenter study of ICU telemedicine reengineering of adult critical care. Chest, 2014; 145(3):500–507; doi: 10.1378/chest.13-1973

Sadaka

, Palagiri

, Trottier

, et al. Telemedicine intervention improves ICU outcomes. Crit Care Res Pract, 2013; 2013:456389; doi: 10.1155/2013/456389

Willmitch

, Golembeski

, Kim

, et al. Clinical outcomes after telemedicine intensive care unit implementation. Crit Care Med, 2012; 40(2):450–454; doi: 10.1097/CCM.0b013e318232d694

Herasevich

, Subramanian

. Tele-ICU technologies. Crit Care Clin, 2019; 35(3):427–438; doi: 10.1016/j.ccc.2019.02.009

10.

Bonnardot

, Liu

, Wootton

, et al. The development of a multilingual tool for facilitating the primary-specialty care interface in low-resource settings: The MSF tele-expertise system. Front Public Health, 2014; 2:126; doi: 10.3389/fpubh.2014.00126

11.

Haranath

, Ganapathy

, Kesavarapu

, et al. eNeuroIntensive care in India: The need of the hour. Neurol India, 2021; 69(2):245–251; doi: 10.4103/0028-3886.314591

12.

Konkani

, Oakley

. Noise in hospital intensive care units–a critical review of a critical topic. J Crit Care, 2012; 27(5):522.e1–522.e9; doi: 10.1016/j.jcrc.2011.09.003

13.

Reynolds

. The tele-ICU: Formative or out-of-date or both? Practice models and future directions. In: Telemedicine in the ICU. ( Koenig

. ed.) Springer International Publishing: Cham; 2019; pp. 3–19.

14.

Krouss

, Allison

, Rios

, et al. Rapid implementation of telecritical care support during a pandemic: Lessons learned during the coronavirus disease 2020 surge in New York City. Crit Care Explor, 2020; 2(11):e0271; doi: 10.1097/CCE.0000000000000271

15.

Macedo

, Garcia

MVF

, Garcia

, et al. Implementation of tele-ICU during the COVID-19 pandemic. J Bras Pneumol, 2021; 47(2):e20200545; doi: 10.36416/1806-3756/e20200545

16.

Munusamy

, Karuppiah

, Bahuri

NFA

, et al. Telemedicine via smart glasses in critical care of the neurosurgical patient-COVID-19 pandemic preparedness and response in neurosurgery. World Neurosurg, 2021; 145:e53–e60; doi: 10.1016/j.wneu.2020.09.076

17.

van der Voort

, de Metz

, Wester

, et al. Telemedicine in a Dutch intensive care unit: A descriptive study of the first results. J Telemed Telecare, 2016; 22(3):141–147; doi: 10.1177/1357633X15590751

18.

Kemp Van Ee

, McKelvey

, Williams

, et al. Telemedicine Intensive Care Unit (Tele-ICU) implementation during COVID-19: A scoping review. Cureus, 2022; 14(5):e25133; doi: 10.7759/cureus.25133

19.

Spies

, Paul

, Adrion

, et al.; ERIC Study Group. Effectiveness of an intensive care telehealth programme to improve process quality (ERIC): A multicentre stepped wedge cluster randomised controlled trial. Intensive Care Med, 2023; 49(2):191–204; doi: 10.1007/s00134-022-06949-x

20.

Moughrabieh

, Weinert

. Rapid deployment of international tele-intensive care unit services in war-torn Syria. Ann Am Thorac Soc, 2016; 13(2):165–172; doi: 10.1513/AnnalsATS.201509-589OT

21.

Kovacevic

, Dragic

, Kovacevic

, et al. Impact of weekly case-based tele-education on quality of care in a limited resource medical intensive care unit. Crit Care, 2019; 23(1):220; doi: 10.1186/s13054-019-2494-6

22.

Ramnath

, Hill

, Schultz

, et al. An in-person and telemedicine “hybrid” system to improve cross-border critical care in COVID-19. Ann Glob Health, 2021; 87(1):1; doi: 10.5334/aogh.3108

23.

Boklage

, Weiss

, Hanefeld

, et al. Telemedicine in emergency responses: Reflections from a critical care telemedicine programme between Uzbekistani and German clinicians during COVID-19. BMJ Health Care Inform, 2023; 30(1):e100675; doi: 10.1136/bmjhci-2022-100675

24.

World Bank. World Bank Country classification: current classification by income. World Bank; 2025.

25.

World Bank. Classification of Fragile and Conflict-Affected Situations-FY25. World Bank; 2024.

26.

Brooke

. SUS: A quick and dirty usability scale. Usability Eval Ind, 1995; 189.

27.

Birkhoff

, McCulloh Nair

, Bald

, et al. Facilitators and challenges in the adoption of a virtual nurse visit in the home health setting. Home Health Care Serv Q, 2021; 40(2):105–120; doi: 10.1080/01621424.2021.1906374

28.

De Cola

, Maresca

, D’Aleo

, et al. Teleassistance for frail elderly people: A usability and customer satisfaction study. Geriatr Nurs, 2020; 41(4):463–467; doi: 10.1016/j.gerinurse.2020.01.019

29.

Dontje

, Kruitwagen-van Reenen

, van Wijk

, et al.; Study Group ALS Home monitoring & Coaching. Evaluation of the nation-wide implementation of ALS home monitoring & coaching: An e-health innovation for personalized care for patients with motor neuron disease. BMC Health Serv Res, 2022; 22(1):1389; doi: 10.1186/s12913-022-08724-6

30.

Seinsche

, de Bruin

, Saibene

, et al. A newly developed exergame-based telerehabilitation system for older adults: Usability and technology acceptance study. JMIR Hum Factors, 2023; 10:e48845; doi: 10.2196/48845

31.

Lezius

, Steinecke

, Herholz

, et al.; U START TICU consensus group. TICU-feedback-tool: Development and pilot application of a questionnaire to assess performance in tele-intensive care collaborations. BMC Health Serv Res, 2025; 25(1):412; doi: 10.1186/s12913-025-12565-4

32.

Teladoc. Teladoc Health_TM Lite 4 User Guide. 2021. Available from: https://manuals.intouchcustomer.com/Current/Online/Lite/Content/Patient%20Devices/Lite/Touch%20Lite%204%20Overview.htm

33.

Bangor

, Kortum

, Miller

. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 2009; 4(3):114–123.

34.

Angus

, Kelley

, Schmitz

, et al.; Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS). Caring for the critically ill patient. Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease: Can we meet the requirements of an aging population? JAMA, 2000; 284(21):2762–2770; doi: 10.1001/jama.284.21.2762

35.

Fonseca

MCM

, Araujo

GTB

, Scorza

, et al. Who is going to turn on the ventilators? Einstein (Sao Paulo), 2021; 19:eAO6211; doi: 10.31744/einstein_journal/2021AO6211

36.

World Health Organization. Consolidated telemedicine implementation guide. World Health Organization; 2022; 36–39.

37.

WHO. Telemedicine: Opportunities and developments in member states. Report on the second global sirvey on eHealth, Vol: 2. WHO; 2010.

38.

WHO. Framework for the Implementation of a Telemedicine Service. WHO; 2016; pp. 45–46.

39.

Meyer

, Benjamens

, Moumni

, et al. Global overview of response rates in patient and health care professional surveys in surgery: A systematic review. Ann Surg, 2022; 275(1):e75–e81; doi: 10.1097/SLA.0000000000004078

40.

, Zhao

KLY

, Fils-Aime

. Response rates of online surveys in published research: A meta-analysis. Comput Hum Behav Rep, 2022; 7; doi: 10.1016/j.chbr.2022.100206

Technical Performance Assessment in an International Tele-Intensive Care Unit Network

Abstract

Background:

Methods:

Results:

Discussion:

Keywords

Introduction

Methods

Study design

TICU technology

Statistical analysis

Results

Discussion

Conclusion

Authors’ Contributions

Footnotes

Acknowledgments

Author Disclosure Statement

Funding Information

Ethics Approval

Consent to Participate

Clinical Trial Registry

Consent for Publication

Availability of Data and Materials

Supplemental Material

Supplemental Material

Supplemental Material

Abbreviations

References