Abstract
Background
Telemedical approaches represent a valuable tool for the management of coronavirus disease 2019 patients, allowing daily clinical assessment, monitoring of vital parameters, remote visits, and prescription of treatment or hospitalization in case of clinical worsening. This cross-sectional study aims to evaluate the use, barriers and facilitators of the “Lazio ADVICE” telemedical platform, a regional system for remote assistance for coronavirus disease 2019 patients at home, according to General Practitioners and Family Pediatricians of the Local Health Authority Roma 1, during the coronavirus disease 2019 pandemic.
Methods
An interview-based survey was performed between December 2020 and January 2021. The survey investigated the demographic information of General Practitioner and Family Pediatricians, the knowledge of the platform, frequency of utilization, usefulness, strengths and weaknesses, and hypothesis of future implementation proposed.
Results
We interviewed 214 physicians and 89 (41.6%) were classified as users and 125 (58.4%) as non-users. Older age and working in District 1, 14 and 15 (vs. District 13) significantly reduced the probability of using the platform physician. Among the 89 users, 19 (21.3%) used the platform every day or even several times a day, 40 (44.9%) several times a week but less than one access per day, 30 (33.7%) used the platform several times a month up to one entry per week. Most of them (92.3%) consider the platform useful. Barriers were poor integration with software and work routine (76.4%), and usability issues (53.9%). Among the 125 non-users, 14 (11.2%) didn’t know the existence of the platform, 60 (48.0%) never tried it and 51 (40.8%) tried to use it. Reported reasons for the interruption of use were not very user-friendly (45.1%), perceived useless (37.3%), non-optimal functioning (23.5%), and lack of time (19.6%).
Conclusion
The pandemic accelerated the implementation of telemedicine services around Lazio Region, starting a positive and continuous exchange of experiences, activities and best practices among physicians.
Introduction
The coronavirus disease 2019 (COVID-19) pandemic is having a considerable health, social and economic impact on the Italian population and the Italian National Health Service (NHS), which, in a very short time, had to necessarily reorganize itself to provide an appropriate response to the renewed health concerns.1–3 Indeed, to contain the outbreak, the Italian Government adopted a series of measures to various degrees of stringency (i.e., social distancing, smart working, mass masking, curfew, business closures, and travel bans) 4 that obliged the NHS to rethink the healthcare service, focusing on alternative ways to provide healthcare assistance, such as telemedicine and other digital tools to deliver healthcare services via remote telecommunications. Telemedicine provides the use of Information and Communication Technologies (ICTs), such as computers and apps for smartphones, to deliver healthcare services at a distance5,6 and allows to remotely exchange information for prevention, diagnosis, treatment, monitoring and evaluation between physicians and patients, 6 proving to be equivalent or more clinically effective (even in terms of mortality) when compared to usual care in many settings.7,8 Finally, in the context of primary care these tools seem to be cost-effective, both for clinicians and the Healthcare System, highlighting how proper use can contribute to the efficiency of the whole system. 9 Despite this, in Italy the use of telemedical applications before 2020 was slightly more than 10%, 10 while many experiences were implemented during the pandemic, 11 following the guidelines of the Italian Government. 5 As with past disasters and emergencies, also the COVID-19 pandemic has prompted the use of telemedicine 12 : indeed, during the pandemic, which imposed a sharp reduction in mobility to limit the spread of the virus and the need to avoid overcrowding of hospitals, telemedicine was an essential and valuable tool for monitoring and controlling symptomatic COVID-19 patients who did not require treatment in a hospital setting. In this way it was possible to assess the patients daily, monitor vital parameters, visit them at a distance and prescribe treatment or hospitalization in case of clinical worsening.13,14
In this context, the Local Health Authority (LHA) Roma 1, following the directive and indication given by the Lazio Region, activated the telemedical platform “Lazio ADVICE/Lazio Doctor'’, to monitor and manage COVID-19 positive patients, and to facilitate the communication between citizens and healthcare professionals through tele-surveillance, tele-visits, and tele-monitoring. 15
LHAs are the institutions that deliver healthcare services to citizens at a local level and are part of the Regional Health Service. In this specific case, the LHA Roma 1 is responsible for the healthcare of more than 1 million inhabitants in a 524 km2 area within the city of Rome. It comprises clinical, technical and administrative departments all serving hospitals and many ambulatory services, the six Health Districts (District 1, 2, 3, 13, 14, 15), which mainly manage primary healthcare and where General Practitioners (GPs) and Family Pediatricians (FPs) work, a Mental Health Department and a Prevention Department, in which the Public Health Service is embedded. 16
Having feedback from GPs and FPs is essential in order to implement an effective telemedicine service not only for the COVID-19, but extended for the management of other conditions, such as frail patients, chronic patients or those living in rural areas.17–19
In this context, this study aims to describe the “Lazio ADVICE—Lazio DOCTOR” platform, adopted by Lazio Region to contrast the COVID-19 pandemic and its implementation in LHA Roma 1 and to evaluate the facilitators and barriers of the platform according to the GPs and FPs of the LHA Roma 1 experience through the submission of an interview-based survey.
Methods
Study design and population
In order to assess the facilitators and barriers of the usage of the Lazio ADVICE telemedical platform, we conducted a cross-sectional study among the GPs/FPs of the LHA Roma 1. In addition, to describe the “Lazio ADVICE—Lazio Doctor” platform an analysis of national, regional, and local laws describing the platform, its implementation and the organizational model was performed.20–23
Survey development and questionnaire structure
A team of researchers and healthcare workers of Districts and the Prevention Department of the LHA Roma 1 developed the questionnaire. Information about age, gender, occupation (GPs or FPs), years of service, LHA district, and the total number of citizens and managed patients were collected by the internal administrative data repository. Those information didn't need to be included in the survey and were likewise available to non-interview subjects.
Overall, the survey was composed of questions (11 and 9 for user and non-user, respectively) investigating the utilization of the platform by GPs and FPs and by their patients, the knowledge and the facilitators and barriers of the platform (Table S1, Supplemental material). A user is defined as someone who has entered the platform at least once in the last 30 days (the day of the interview is the last day considered). Thus, specific questions per user and non-user are proposed, such as knowledge of the platform, frequency of utilization, usefulness, strengths and weaknesses and hypothesis of future implementation offered by Lazio Region. The estimated time to complete the phone interview is about 5–10 min, including the time required to explain the platform to the physician.
Sample size determination, sample selection, and randomization
To determine sample size, the population size was considered. As of October 30, 2020, the total number of LHA Roma 1 GPs and FPs is 984 (854 GPs and 130 FPs). GPs and FPs are unevenly distributed among the six Health Districts, according to the dimension and population of each District (Table S2, supplementary material). This parameter was considered in the sample size determination to not lose the strength of the information obtained on a district level considering the margin of error (MOE) and the confidence level (CL). The MOE indicates the likelihood that the survey results represent the whole population being studied; thus, a lower MOE indicates more reliable results. The CL is a percentage that indicates the confidence that the population would give a response within a specific range. Thus, Epi Info software was used to calculate sample size, to have a number of responses per District with at least 80% CL and no more than 10% MOE (Table S2, Supplemental material). Thus, based on the sample size calculation, the minimum target of 196 subjects in all the LHA to be interviewed was obtained. This number was close to 20% of the whole Roma 1 LHA's GPs/FPs population (
Survey dissemination and GPs and FPs enrollment
The interviews took place simultaneously in all 6 districts in December 2020, preceded by an invitation email describing the initiative sent by the medical directorate of LHA Roma 1. Three medical doctors led the administration of the questionnaire. The survey concluded at the end of January 2021, with a total duration of about 2 months. The participation of GPs and FPs was voluntary and unpaid. Informed consent was requested from all the participants, and it was necessary to participate in the survey.
Statistical analysis
We performed descriptive analysis, calculating the median, 25° (Q1) and 75° (Q3) percentile, interquartile range, the mean and standard deviation for each response at LHA and district level and inferential statistic tests (Chi-square test,
Checklist statements
This study was prepared in accordance with the STROBE statement checklist—cross-sectional studies. 24 The checklist is available in the Supplemental material. In addition, we described the intervention following the Template for Intervention Description and Replication (TIDieR) checklist and guide. 25 More information about the application of the TIDieR checklist in our context is provided in Supplemental materials.
Results
The Lazio ADVICE—Lazio Doctor platform
The “Lazio ADVICE—Lazio Doctor” platform represents the regional tool used to manage COVID-19 patients. It has been implemented and improved during the pandemic by the Information Technology Service of the Lazio Region,20,21 supported by a working group of the Regional Healthcare Service and all LHAs guided by a project manager identified by the Lazio Region. 22 The system is implemented for patient monitoring in multiple care settings made by the GPs and other healthcare workers (HCWs) involved in the pandemic management. The platform also consists of a corresponding patient smartphone application called “Lazio Doctor COVID,” in which the users can insert and collect personal data and information about their healthcare status and share them with their GP. Moreover, patients could complete a questionnaire that investigates COVID-19-related symptoms and epidemiologic data such as contact with a COVID-19-positive case during the previous 14 days or a possible trip taken in a country with a high incidence of new cases. Additional information about the platform can be found in the supplementary material. Data management of patients resident or domiciled in the area of competence of the LHA was performed with respect to the requirements of the current privacy laws in Italy and of the European General Data Protection Regulation. 26
The LHA Roma 1 organizational model
To manage the new telemedicine platform made available by the Lazio Region, the LHA Roma 1 implemented a widespread model involving HCWs of the Healthcare Directorate, Public Health Service, Districts, ICTs and Pharmaceutical Service. Each service identified a contact person responsible for the platform, always connected with the other members of the task force and GPs and FPs. In addition, a one-day training course was held for all the LHA contact persons, GPs and FPs to illustrate the functions and management methods of the service (Figure 1).

The organizational structure of the Local Health Authority (LHA) Roma 1 to manage the new telemedical platform “Lazio ADVICE/Lazio Doctor” made available by the Lazio Region.
Survey results
Comparison between interviewed and non-interviewed subjects
The survey administration lasted around one month and was led by three medical doctors. At the end of January 202 l, the sample target for each district (80% CL and 10% MOE) and the whole LHA (90% CL and 5% MOE) was reached. 214 subjects (21.8% of the total population) were interviewed. Characteristics of the interviewed sample, as compared to the remaining population are described in Table 1, while more details on the number of interviewed subjects per District are described in Table S3, Supplemental material.
Description of the interviewed sample, as compared tothe remaining population.
GPs: General Practitioners; FPs: Family Pediatricians. In bold
4 missing
32 missing
Comparison between user and non-user
Among the 214 respondents, 89 (41.6%) were users, and 125 (58.4%) were not users, with statistically significant differences between districts (Table 2). There was no statistically significant difference in the number of patients per physician between users and non-users (
Comparison between user and non-user.
In bold
*It refers to the median number of patients managed by each physician
Finally, more than half of the sample (57.3%) had been medical office owners as GP/FP for more than 20 years, 28.2% from 5 to 20 years, and 14.5% for less than 5 years (Table 3). The proportion of physicians using the platform appeared to vary with years of practice (
Combination of platform usage and years of practice (firstly considering it as separate, then analyzing less or equal to 20 years and more or equal to 5 years as unique variable, respectively in the second and in the third option).
In bold
Performing the logistic regression model to ascertain the effects of age, the number of managed patients, years of practice, job description and working districts on the likelihood that physicians use the Lazio ADVICE platform, no variable with a value of VIF >10 was detected; the highest value was 1.44 for age (years). The logistic regression model was statistically significant, χ210 = 32.8,
Logistic regression predicting the likelihood of being a platform user.
Note. Estimates represent the log odds of “Lazio Advice user = 1” vs. “Lazio Advice not-user = 0.” GP: General Practitioner; FP: Family Pediatrician. In bold
User facilitators and barriers
Among the 89 users, 19 (21.4%) used the platform every day or even several times a day, 40 (44.9%) several times a week but less than one access per day, and 30 (33.7%) used the platform several times a month up to one entry per week. The comparison of the medians of the number of managed patients from these three groups but referring only to GPs was not significant (

Frequency of platform use by general practitioners. Every day, even several times a day: Median of managed patient 1542 (Q1 909,75; Q3 1550,5). Several times a week, but less than one access per day: Median 1072 (Q1 775; Q3 1284). Several times a month, up to 1 access per week: Median 1031 (Q1 389; Q3 1421,5).
Non-user facilitators and barriers
Among the 125 non-users, 14 (11.2%) did not know the existence of the Lazio ADVICE platform, 60 (48.0%) never tried to use the platform, and 51 (40.8%) tried to use it.
The most reported reasons for the interruption of use were not very user-friendly (45.1%), perceived useless (37.3%), non-optimal functioning (23.5%), and lack of time (19.6%).
The most reported reasons for not using the platform among those who were aware of its existence were: Lack of time (43.3%), perception of uselessness (30.0%), use of another tool (26.7%), difficulty in accessing, or unwieldy referral by colleagues (15.0%).
Discussion
Our study describes the platform “Lazio ADVICE—Lazio DOCTOR” to manage COVID-19 patients and evaluates the facilitators and barriers according to a sample of GPs and FPs of the LHA Roma 1.
Out of 214 subjects, less than half used the platform, although more than 92% found it useful for managing COVID-19 patients. Thus, this contrast seems to suggest that many barriers limit the use of these tools.
Although digital tools (such as telemedicine, patient portals, and smartphone applications) aim to improve the quality of care, the contextual factors, such as infrastructures, human resource management, financial resources, and leadership styles, influence the utilization of these technologies 27 and impact on health outcomes, system efficiency, and patient satisfaction.28,29 The COVID-19 pandemic showed the need to use these tools, which should be completely integrated into healthcare systems. Specifically, during the pandemic, digital tools, such as telemedical platform, aim to bring medical services to isolated patients to ensure monitoring and treatment without physical contact to minimize the risk of transmission. 14 Despite the importance of this tool and the significant increase in its use during the pandemic, some barriers limit its use, both in HCWs and patients.30–37 For example, a 2016 systematic literature review investigating barriers to adopting telemedicine worldwide identified the issues with technically challenged staff, resistance to change, cost and reimbursement, age, and the level of education of patient. 35 Moreover, many other concerns were identified by HCWs, such as the lack of a hands-on physical exam and physical distancing, the potential distractibility of patients due to competing responsibilities at the time of the telemedical visit, the need to retrain staff, the privacy issues, the inequalities in access and the use of telemedicine related to demographic (e.g., older or disabled individuals) and socioeconomic conditions.34,36 In addition, other barriers, highlighted during the pandemic, that prevent telemedical approaches from being integrated into common practice, are the clinician willingness, the financial reimbursement, and the organization of the health system. 12 However, in our situation, we did not detect the issue of reimbursement as a barrier to use, due to the different structure of the NHS, that do not provide a financial remuneration for the GP or FP's use of the ADVICE platform to monitor and manage COVID-19 positive patients. Specifically, in our study, the main barriers were time to spend on the activity, knowledge of the platform, usability, and interoperability issues with other digital systems. Similarly, in a German study that assessed the acceptance of telemedical tools in a sample of rheumatologists and GPs, the major obstacles encountered were the purchase of technology equipment, administration, and inadequate remuneration. 38 On the other hand, Bos et al. 39 found that the main barriers were the impossibility of performing a physical examination, difficulty estimating how the patient is doing and difficulty in reaching patients. In our results, instead, users (especially those with a high number of patients) considered useful the platform to reach and manage COVID-19 patients by monitoring their health status. Another important barrier is the age of the GPs/FPs: In our study, we found that younger physicians were more compliant with the use of the platform, in line with what has been observed in other studies in which the age of the provider increases, the use of telemedicine decreases.40,41 These results can be related to the concept of the digital divide, “the gap between those who have access to and use ICTs including internet connectivity, internet-enabled devices and digital literacy skills and those who do not”. 42 Although the Lazio Region has given equal opportunities to all physicians, many GPs preferred to use the video call systems present on the telephone rather than a more structured (i.e., secure and reliable) telemedicine system. An important consideration concerns governance and organization at the central level. Two factors are needed to implement a successful intervention with high adherence to telemedicine services: strong leadership at the central level change-management techniques and education and information of physicians.16,19,35 Our study, also through the logistic regression analysis, showed the highest adherence of usage in District 13 where the corporate manager of the “Lazio ADVICE—Lazio DOCTOR” project works. Thus, this finding supports the idea that solid central leadership can play a crucial role in the use of technology through persuasion, direct involvement of physicians, and targeted training interventions. Similarly, the health Directorate implemented information and education programs on the use of this telemedicine service that should be included into structured educational and training programs. 12 Still, these will need to be implemented to involve more physicians, especially given new epidemic waves. 43 The use of telemedical platforms by GPs do not aim to replace in-person visits, but to expand and support the common medical practice following the principles of evidence-based medicine, even more during health emergencies. 12
Our study has several limitations. First, the survey was addressed to physicians in only LHA Roma 1, an LHA that insists on a predominantly urban environment. However, our sample size determination allowed us to have a homogeneous distribution of physicians across the territory, with statistical representativeness for each district. Thus, our results would be scalable to other comparable LHAs in Italy. Another limitation is using a questionnaire not previously validated in the literature, although we conducted a pilot survey with a small sample of physicians that showed high reliability. Finally, we did not consider patients’ opinions, which may be a future study project, as well as reevaluate adherence to the use of the digital tool during subsequent pandemic waves. Lastly, for the development and evaluation of a complex intervention such as “Lazio ADVICE,” a mixed-methods design would be a useful methodology for better-understanding people's opinions, and it will be considered in the development of future applications of this tool. 44
Conclusions
Lazio Region has adopted innovative strategies to develop an adequate response to the COVID-19 pandemic, such as “Lazio ADVICE—Lazio DOCTOR.” The pandemic has accelerated the implementation of telemedicine services around the Region, starting a positive and continuous exchange of experiences, activities and best practices among all LHAs. The pandemic should be an opportunity to improve the use of this tool and its application to other chronic disease management settings as well, but of course it has to be competitive, offering something different or additional to teleconsultation.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076231174099 - Supplemental material for The “Lazio ADVICE” telemedicine platform: First results of general practitioners’ usage, facilitators and barriers in the Local Health Authority Roma 1
Supplemental material, sj-docx-1-dhj-10.1177_20552076231174099 for The “Lazio ADVICE” telemedicine platform: First results of general practitioners’ usage, facilitators and barriers in the Local Health Authority Roma 1 by Andrea Barbara, Leonardo Villani, Paolo Lombardo, Paolo Parente, Antonella Gemma, Debora Angeletti, Tiziana Chiriaco, Antonio Mastromattei, Svetlana Akselrod, Mauro Goletti, Enrico Di Rosa and Corrado De Vito in DIGITAL HEALTH
Footnotes
Acknowledgments
The authors would like to thank Dr Emanuela Maria Frisicale, Dr Giulia Tammam, Dr Leonardo Simonetti, and Dr Alessio Perilli for their support.
Author contributions
All authors contributed to the study’s conception and design. Material preparation and data collection were performed by Andrea Barbara, Antonella Gemma, and Tiziana Chiriaco. Andrea Barbara and Paolo Lombardo performed the statistical analysis. The first draft of the manuscript was written by Andrea Barbara, Leonardo Villani, Paolo Lombardo, and Paolo Parente. Debora Angeletti, Antonio Mastromattei, Svetlana Akselrod, Mauro Goletti, Enrico Di Rosa, and Corrado De Vito commented on the latest version of the manuscript. Mauro Goletti and Corrado De Vito supervised the study. All authors read and approved the final manuscript.
Data availability
Data are the property of the LHA Roma 1 and are accessible under reasonable request.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The study was approved by the “Comitato Etico Lazio 1”, the Lazio Region Ethical Committee, protocol number 1022/CE Lazio 1.
Funding
The authors disclosed no financial support for the research, authorship, and/or publication of this article.
Guarantor
Paolo Lombardo.
Informed consent
All participants gave their consent to participate to the study.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
