Abstract
COVID-19 pandemic impacted the world dramatically and how communities and societies behaved. The purpose of this study was to explore lecturers’ perspectives on emergency remote teaching, not only from technology-prone lecturers but encompassing contributions from disparate academic fields, which brought assorted experiences into the data. The qualitative study analyses the perspectives of 344 lecturers from 35 higher education institutions in Portugal instigated by the COVID-19 outbreak. The thematic analysis revealed three themes related to the advantages and disadvantages: the lecturing process, learning process, and life and society. Regarding the concerns, four themes emerged: learning process, self-regulation and compromising the future, resources, the assessment process, and well-being. Finally, solutions encompass three themes: pedagogy and innovation, normative framework, and digital literacy. With a special focus on the solutions, all these themes contribute to designing the structure of a training program for the 21st-century higher education lecturer, emphasizing the digital domain. The themes cluster in digital-based innovative pedagogies (mastering digital education software, active pedagogies for digital environments, lecturer posture in digital environments, “for-digital” studying material production, e-proctoring, and transferable competences besides the digital), and digital competences (transgenerational digital-base communication, webcam behavior, well-being in digital settings, and general digital competences).
Plain Language Summary
The global impact of the COVID-19 pandemic significantly altered societal behaviours and academic practices. This study delved into the perspectives of 344 lecturers across 35 higher education institutions in Portugal concerning the sudden shift to emergency remote teaching prompted by the pandemic. It aimed to understand their varied experiences and viewpoints, encompassing diverse academic disciplines. Through qualitative analysis, several key themes emerged. Lecturers highlighted both advantages and disadvantages in three main areas: the teaching process itself, the learning experience for students, and the broader impact on life and society. Concerns were expressed regarding the learning process, potential compromises to future educational outcomes, resource limitations, assessment methods, and personal well-being. Moreover, the study uncovered suggestions for potential solutions across three overarching themes. These included proposals for innovative pedagogical approaches, the establishment of normative frameworks to guide remote teaching, and an emphasis on enhancing digital literacy among educators. Notably, lecturers underscored the importance of incorporating digital-based innovative pedagogies, such as mastering educational software, adopting active teaching methods for digital environments, understanding proper lecturer conduct in online settings, creating materials suited for digital learning, implementing e-proctoring, and fostering transferable competencies beyond digital skills. Additionally, lecturers emphasized the significance of digital competencies, spanning transgenerational digital communication, appropriate webcam usage, well-being considerations in digital learning spaces, and overall proficiency in digital tools and platforms. These suggestions collectively offer a roadmap for crafting a comprehensive training program geared towards 21st-century higher education lecturers, placing a significant emphasis on digital integration and proficiency in educational settings.
Keywords
Introduction
A Wild Card is “a future development or event with a relatively low probability of occurrence but a likely high impact on the conduct of business” (BIPE et al., 1992, p. v; Petersen & Steinmüller, 2009, p. 1). COVID-19 was undoubtedly a wild card with pervasive impacts in many fields, one of them in higher education lecturing. Due to the lockdowns, many higher education lecturers were suddenly forced to teach classes outside the usual classrooms supported by digital communication software, a situation coined as emergency remote teaching (ERT). ERT, however, is not like traditional, purposefully designed online teaching. ERT refers to courses designed initially for in-person instruction that have been modified for online education due to emergency or crisis circumstances (C. B. Hodges et al., 2020) and “is not to re-create a robust educational ecosystem” (C. Hodges & Fowler, 2021, p. 119). Indeed, ERT presented new and unique challenges for teachers who were required to adapt rapidly to the dramatically different digital teaching and learning environment (Xie et al., 2019).
Lecturers from distinct fields and with assorted technology proneness were dragged into ERT, both those already acquainted with e-learning or blended learning LMS-based education and those whose courses are based on paper books and expositive discourse. After some time experiencing ERT, what was their perspective? What knowledge and techniques did the former find more pertinent for ERT, and what discoveries did the latter find helpful to overcome the problems? COVID-19 was a wild card that provided countless disparate experiences. Following the implicit suggestion in Bozkurt and Sharma (2020), the present study’s purpose is to gather the maximum amount of lecturers’ feedback focusing on the advantages, disadvantages, concerns, and solutions and contribute to outline the competences for the higher education lecturer in the 21st century.
COVID-19 Pandemic and the Development of ERT
COVID-19 has forced universities worldwide to interrupt in-person teaching to reduce person-to-person physical contact and the spread of the virus (Moorhouse & Kohnke, 2021). Therefore, online-only modes of instruction, known as emergency remote teaching (ERT), have been adopted (Bozkurt & Sharma, 2020; C. B. Hodges et al., 2020). ERT is different from online teaching. Unlike ERT, online education results from careful instructional design and planning (C. Hodges & Fowler, 2021) and requires an investment in an ecosystem of learner supports, which takes time to identify and build (Moorhouse & Kohnke, 2021). As pointed out by Ali (2020), the development of online teaching is a technical issue and a pedagogical and instructional challenge that requires extensive preparation in terms of the choice and design of teaching and curricular material and requires extensive knowledge of online instruction. The strengths, weaknesses, opportunities, and challenges of using online learning in past natural disasters have been analyzed by Dhawan (2020, p. 14), who emphasizes “technical difficulties, time management, distractions, frustration, anxiety and confusion, lack of personal/physical attention, quality of education, digital illiteracy, technology cost,” among others, in what it concerns weaknesses and challenges.
In Portugal, the context of this current study, due to COVID-19, many higher education institutions (HEIs) were forced to move from face-to-face classroom instruction to ERT. As a result, all activities that included academic staff physical presence were suspended in late March 2020 (Ministro da Ciência Tecnologia e Ensino Superior, 2020), and lecturers had to face challenges and adapt to teleworking without careful preparation. In a short time, the academy community started to teach in front of a computer screen. Rapanta et al. (2020, p. 926) stated that “teachers have, almost overnight, been asked to become both designers and tutors.”
Therefore, it is essential to explore the perspectives of lecturers from HEIs regarding how they have been coping with the shift to ERT since it can impact the quality of their teaching, mental health, and long-term commitment to education as well as students’ learning (Carson & Chase, 2009; Ryan & Deci, 2020). Furthermore, the short evidence of similar situations in the past makes it hard to anticipate the impact of these changes on the academy community. All the circumstances where HEIs had to develop online classes differed from any other registered event nationally or internationally.
Overview of the Previous Studies on ERT
Literature in this field has evolved, and most of the previously published studies focus on the impact of ERT on HEIs. Even though the move towards ERT “was not easy neither for instructors nor for students who were not prepared for this shift” (Serhan, 2020, p. 340), most previous studies still adopt the student’s perspective. For example, some studies conducted with undergraduate students found ERT difficult because, at home, many distractions affect their focus and understanding of the lessons (Ag-Ahmad, 2020; Li, 2022). In the study of Rakhmanina et al. (2021), the most challenging skill perceived by the students is listening, and the lack of physical interaction is demotivating for them. On the other hand, Hazaymeh (2021) found multiple challenges experienced by students, namely technical and internet connection problems and no physical interaction, leading to demotivation and a lack of technological tools necessary for effective online learning.
The research conducted on the lecturer’s perspective in HEIs highlights some of the most challenging issues, such as the lack of teaching experience through online platforms, lack of support from educational technology teams, and difficulties in accessing the internet (Daniel, 2020; C. Hodges & Fowler, 2021; Zhang et al., 2020), the inevitability of remaking of educational experience, the weakness of the online teaching infrastructure, the inexperience of teachers, the information gap (Zhang et al., 2020), the inability to teach and learn practical and clinical work, and lack of immediate feedback for students (Mukhtar e tal., 2020). Moreover, it is important not to neglect the complex environment at home (Kim, 2006) in which unexpected appearance or interruption of family members, friends, and or pets may cause disruption or diversion of online learning participants’ attention during the online teaching and learning process (Adedoyin & Soykan, 2023).
Kraft et al. (2021) also examined teachers’ experiences during ERT with a sample of 7,841 teachers across nine states of the United States. Teachers reported various challenges related to their skill to manage their professional responsibilities, concerns, and difficulties in balancing family and professional life. They found differences depending on the number of years of experience concerning conciliation problems. Half of the mid-career teachers reported that caretaking responsibilities made their job difficult. Late-career teachers reported being less comfortable teaching online. Still, at this level, a group of teachers called the “surviving’ group”—identified by Moorhouse and Kohnke (2021) questioned their ability to teach, found it hard to build a rapport with learners, felt isolated, and struggled to find job satisfaction. Adedoyin and Soykan (2023) added the socioeconomic factor as an additional challenge for university teachers. Students with low socioeconomic backgrounds found it difficult to migrate as early as expected since they could not attend school due to the pandemic. Fishbane and Tomer’s (2020) report on what students with no internet access are to do during this COVID-19 pandemic also shows that as the level of poverty increases in the community, the rate of internet accessibility declined rapidly and by implications, students with no or low socioeconomic power to afford broadband connection are most vulnerable to fall behind or encounter additional challenges to meet up with others in online learning which is supported by other authors (Adedoyin & Soykan, 2023; Cullinan et al., 2021).
In summary, studies overall reveal several technological, pedagogical, and social challenges related to ERT (Adedoyin & Soykan, 2023; Ferri et al., 2020). The technical challenges are mainly associated with the unreliability of Internet connections and many students lacking mandatory electronic devices. The pedagogical challenges are primarily associated with teachers’ and learners’ lack of digital skills, the lack of structured content versus the abundance of online resources, learners’ lack of interactivity and motivation, and teachers’ lack of social and cognitive presence. That is, the ability to construct meaning through sustained communication within a community of inquiry (Garrison et al., 1999; Stewart, 2017). Lastly, the social challenges are mainly related to the lack of human interaction between teachers and students as well as among the latter, the lack of physical spaces at home to receive lessons, socioeconomic inequalities among students, and the lack of support from parents who are frequently working remotely in the same spaces (Ferri et al., 2020).
Studies have also explored university teachers’ perceptions of the advantages of using ERT during COVID-19. For example, the qualitative focus group study of Mukhtar et al. (2020) conducted in medical and dental institutes in Pakistan reports advantages such as remote learning, comfort, and accessibility. Moorhouse and Kohnke (2021) also found that for a group of teachers—called a “thriving group”—the experience with ERT reinvigorated their teaching and provided them with new skills. Dhawan (2020, p. 10) emphasizes the opportunity to drive education improvements forward with ERT, namely to “scope for innovation and digital development,” to “design flexible programs,” and to “strengthen skills,” particularly on “problem-solving, critical thinking, and adaptability.”
Babinčáková and Bernard (2020) and Adedoyin and Soykan (2023) also systematized, in a conceptual study, the opportunities generated by the ERT in the era of the COVID-19 pandemic. Such opportunities entail research and technological innovations, as well as socioeconomic interventions. Research innovation relates to the need for instructional technologists, especially distance education researchers, to meet the latest challenges of online learning. These research advancements should cover, for example, the need to review the digital transformation process of HEIs or design an online learning model that will reduce the workload on the instructors (Adedoyin & Soykan, 2023). Technological innovations related to the efforts of universities and other research centers worldwide to provide research avenues for researchers’ collaboration to produce positive results as early as possible for the prevention and control of the pandemic. For example, some universities in North Cyprus have made several scientific innovations (e.g., “3D Multiplexer Ventilator”) to assist health workers to eradicate the pandemic and to the public to stop the rapid spread of the virus (Kativhu, 2021). Lastly, socio-economic interventions relate to the efforts of university communities in providing socio-economic supports to the students (i.e., food items, stoppage of increment in tuition debt policy on students, psychological, and medical assistance to students and residences). Some HEI across the globe also keyed into such socio-economic intervention as the form of corporate social responsibilities and service innovation to provide free internet for their students and faculty (Cullinan et al., 2021; Ogunmokun et al., 2022).
Even though the literature on ERT and its impacts on HEIs has evolved due to COVID-19, studies predominantly focus on the student’s perspective rather than the lecturer’s perspective. Moreover, those focused on the lecturer’s views are primarily conceptual, and qualitative studies are quite scarce. To the best of our knowledge, the few qualitative studies found employed predominantly case studies approaches and focus groups.
Research Purpose and Questions
The primary purpose of this study is to explore lecturers’ perspectives on ERT in response to COVID-19. The research approach is qualitative in its nature and addresses the following research questions:
What are the advantages and disadvantages identified by lecturers regarding ERT?
According to the lecturers, what concerns do they encounter regarding ERT?
What are the solutions identified by lecturers with respect to ERT?
Methods
The study is qualitative, based on open-ended questions deployed in the web, and delivered to lecturers living in Portugal over one month between 19 June and 30 July 2020 after classes were over, during the national alert period. The web data collection form was disseminated through the institutional web pages (Faculty of Medicine of the University of Porto and the Centre for Research in Health Technologies and Services) and social media platforms (Facebook, Instagram, LinkedIn). It was conducted in line with the Declaration of Helsinki. It received approval from the Ethics Committee of Health Ethics Committee (CES) of the São João Hospital Centre/FMUP (Ref CE 98-2020 on 29 May 2020).
All participants gave their online informed consent with mandatory acceptance to proceed. When accessing the link, participants were presented with an introduction with the study purposes, duration of the data collection and guarantees of anonymity and confidentiality.
Participants
The participants in the study were lecturers working in Portugal when the COVID-19 pandemic started. It is a voluntary response sample because the respondents freely answered to the online invitation, or not. A total of 344 lecturers from all over the country participated in this study, which sociodemographic characterization is reported in Table 1.
Sample Characteristics of Participants (
Data reveals that 33% do not identify the education areas where they are affiliated; 32% of participants worked in health area; 10% in tourism; 7% in social service; 7% in accounting; 3% in arts; 3% in business; 3% in engineering; 3% in arts; and 2% in other areas (e.g., architecture).
Open-Ended Questionnaire
The study included sociodemographic and contextual data (e.g., gender, age, marital status, information regarding the existence of children, educational qualifications, number of years of professional experience, technological devices, previous teaching experience in distance learning, etc.). At the end of this section, four open-ended questions were presented, where subjects had the opportunity to freely elaborate about the four topics advantages, disadvantages, concerns, and solutions, as follows:
○ Advantages: “Regarding the distance teaching-learning process, indicate its advantages, please.”
○ Disadvantages: “Regarding the distance teaching-learning process, indicate its disadvantages, please.”
○ Concerns: “What are your concerns about the impact of distance education on student learning?”
○ Solutions: “Regarding the distance teaching-learning process, please indicate solutions that might overcome possible damages.”
The phrased answers to the open-ended questions were analyzed according to the six-phases method Thematic Analysis (Braun & Clarke, 2006) supported in Nvivo R1 (https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home). Phases 2 through 5 were implemented with two data analysts working in tandem, that is, in series but separately.
Firstly, data were imported into Nvivo. After perusing, one data analyst generated the initial codes in a straight list. The other data analyst collated the list with the codes into subthemes and themes and reviewed them, matching the original subjects’ responses. Then, the second data analyst named both subthemes and themes. These roles were reversed among the four open-ended questions. Next, both data analysts met to clear up pending divergences, agreeing on the hierarchical organization of themes, subthemes, and grounding codes. By the end of the fifth step, a third analyst reviewed the collating of the codes and names of themes and subthemes. Finally, the analysts wrote memos for each theme and then, overall, for each open-ended question, describing the noteworthy aspects that emerged from the data.
Results
Advantages and Disadvantages
Table 2 depicts the themes, subthemes, and aspects that emerged from the qualitative data analysis of the open-ended questions Advantages and Disadvantages. The three themes are the lecturing process, the learning process, and life and society.
Themes, Subthemes, and Aspects Emerged From the Data Analysis in the Open-Ended Questions Advantages and Disadvantages.
Lecturing Process
The theme lecturing process focuses on the lecturer and institutional side and the instructional and delivering processes. It encompasses three subthemes: pedagogic strategies, class management, communication and interaction, and digital literacy. Several aspects, both considering advantages and disadvantages, are framed by the subthemes.
Regarding the advantages of the pedagogic strategies, subjects referred to ERT as an opportunity for pedagogic innovation, improving lecturing in new ways, together with the studying materials, and enhancing the content structure. The technological support of the classes makes it easy to invite guest lecturers or practitioners, contributing to the enrichment of the classes. Besides these advantages, the studying materials, and even the classes, may be freely available to the students which helps them on their progress, either because they missed the class for some reason, or because they missed attention in a moment, or even to return to a specific explanation in the future for better comprehension. For instance, There is the possibility of including pedagogic materials, like videos, etc., and involving speakers from other institutions, even geographically distant.
On the negative side, the forced move to the online has disadvantages. One aspect that is often addressed is practical classes. According to the respondents, online educational technologies did not find a suitable way to deliver practical classes efficiently. That is the case of laboratories, internships and, even more stressed, the students of medicine courses who need to contact with patients and extensively deal with clinical cases: Students are sharply demotivated because of the missing practical classes where they may put theoretical subjects into practice since most of the learning encompasses equipment handling and challenges all the senses to be balanced with the theory.
The difficulties with assessing students are another issue in ERT because of the problem to control fraud. Another disadvantage in the pedagogic strategies realms is course material preparation, which is now time-consuming. Such materials may be a shortcut or, so to maintain the quality patterns, the lecturer must invest more time, either in studying materials production or learning the technological tools to adapt to digital support.
The second subtheme, class management, communication, and interaction, is extensively addressed either by the advantages side or disadvantages. Class dynamics, productivity, and students’ concentration and participation contribute to class effectiveness. The online shape the students’ behavior in the class, improving it, which facilitates the flow of the communication, which, in turn, helps in the management of big classes. Improved communication extends out of the classroom. It is easier to schedule meetings and have individual tutorial sessions based on digital platforms. Such strategies help to keep proximal with students. Although, there are also disadvantages due to the online. For instance, they find it difficult to obtain feedback from students, perceive their learning outcomes and involvement, or have difficulty managing discussions and debates during the class. Individual monitoring became a challenge, and only proactive students may take advantage.
Still in the class, some respondents find the online platforms inappropriate to manage big classes or work with groups of students. It is difficult to keep the pace in big classes (c. 40 students) and, mainly, the work in groups that is being unfolded.
The lecturing quality is impaired, for example, because supplementary activities are reduced, or limited, or even impossible to run in ERT. Keeping up assignments and students’ progress, or the absence of proper lecturing methods are other issues that compromise quality: The structure of knowledge transmission to the students (where knowledge is pushed into their heads) does not allow the use of the physical presence pressure to guarantee that students keep the focus on the task.
Finally, the lack of personal contact with students compromises interaction, which harms the communicative flow necessary for lecturing, that is, all the process is at high risk: It misses the verbal communication to follow students’ reasoning, preventing what research advocates about the lecturer need to understand what is missing in each student so he may progress in learning.
Digital literacy is the third subtheme. On the former side, ERT pushed lecturers’ digital progress; they just had no other way. However, it is important to say that FCT (the Portuguese governmental foundation for research) support deploys free Zoom access to most HEIs, which may have facilitated progresses in this field. Despite the improvements, the coin has another side. The demands over the lecturers’ shoulders forced them to adapt to online lecturing and extra work. The additional requirements extend to the technological means and devices with the lecturers having to use their own hardware and an internet connection, considering that an important share of them has problems with digital proficiency (e.g. so far had no idea about what a VPN is) or even empathy with technology, besides the common issues about technology (e.g. low-speed internet or even dropping often, sharing the screen or the presentation that does not enter, the computer does not recognize the webcam or the microphone, etc.): The current software does not comprise adequate tools for easy interaction between groups or pairs student/lecturer. They work for videoconferencing, but we need a new platform for classes.
Learning Process
The theme learning process collates respondents’ contributions to the student and the learning process. It includes three subthemes: lifelong key competencies, communication and interaction, and student involvement.
Beyond technical competencies, 21st-century citizens should learn and take into practice other skills, for instance, vd. the European Commission’s “Recommendation on Key Competences for Lifelong Learning” (https://ec.europa.eu/education/education-in-the-eu/council-recommendation-on-key-competences-for-lifelong-learning_en) (European Commission & Directorate-General for Education Youth Sport and Culture, 2019). The respondents find advantages and disadvantages in ERT, sometimes contradictory. On the advantages side, ERT contributes to promoting digital competences and individual development as a whole, and punctuality, autonomy, creativity, and accountability more specifically: Familiarisation/contact with digital tools, which, in another context, would not be necessarily explored. [ERT] allows a more autonomous work of the student granting the acquisition of competences of other kinds. [ERT] requires increased responsibility and organization in the student in what it concerns the [learning] process.
Other respondents find that ERT limits the enhancement of key competencies in students and stresses difficulties in self-organization. The ERT environment turns students less spontaneous and takes them to lose interest in synchronous activities. In parallel, students have excessive workloads, and, in ERT, it is challenging to control students’ attendance: (…) compromises the good development in the students of the social and intellectual competences, once that [ERT] does not offer differentiated stimuli or the contact with work platforms other the digital. (…) and, more important than anything, there is no space for the development of the so important transferable competencies like teamwork, socialization, communicating with peers, etc.
On the student learning side, the subtheme communication and interaction stems from ERT’s positive and negative aspects. While students’ learning may be improved because ERT favors support among students and student’s opportuneness (for instance, because of no wasted time with travelling and digital-based connections may be taken immediately from home), one also must consider that it precludes the natural interaction among students and their psychology should suffer from the new environment.
Student involvement, the third subtheme, is pertinent, even critical, to the learning process, and the respondents contribute extensively to this field. ERT, and digital platforms in general, contribute to facilitating mutual support among students, given that the elimination of travelling freed time to invest in study and progress. Through ERT, students also contribute to class effectiveness, bringing dynamics with their participation and improved concentration, as there are fewer distractors. Those who expect to be attentive to the class succeed because those who chat with colleagues and are distracted do not disturb [the class]. Some students are more expressive and participate in teaching activities because they feel safer and less observed by their peers. Students’ concentration on the activities got better.
However, according to the respondents, students’ involvement has room for improvement. Students ill-use the webcam, or it is even turned off. Because of digital mediation, it is difficult for the lecturer to motivate students to participate in the class. It is also difficult to increase their attention and concentration on the relevant matters. Besides, there are psychological aspects. The digital facilitates distance, which may lead students to give up on university. Furthermore, the distance may raise awareness of students’ self-isolation. (…) especially in daytime classes, some remote classes are a bit demotivating for the lecturer, as students do not turn on their webcams, and the lecturer feels “being like talking to a brick wall.” There is no reaction to the contents that I present. I would prefer to record a video. It seems that I am talking alone. It is worth noting that some students, despite being contacted in several ways, get completely disinterested in the class, in most cases, because they could not adapt to distance learning yet.
Life and Society
The lecturers who participated in the open-ended questionnaire also contributed to aspects of life, in general, and society. This theme comprises three subthemes: lifestyle, inclusion, and public health.
ERT, and digital-based teaching-learning in general, offers a means to keep outbound students proximal due to the facilitated and affordable communication. On the lecturers’ side, there is increased comfort: better nutrition, fewer displacements and travelling and, therefore, less stress and less environmental costs, together with saving money and time; saving time means more time for research. It is important to emphasize that there were 134 contributions of the respondents that directly addressed the aspects related to fewer displacements and travelling and time and money savings. [The advantages of ERT are:] safety/comfort of the place chosen for telecommuting; economic (savings in fuel); time (less time wasted on travelling); easier access to assorted documents. (…) there is time for better and more adequate eating.
Nonetheless, the respondents noticed that many students lack proper space for taking the classes. On the lecturers’ side, there are impacts over life organization and negative psychophysiological aspects to take into account, for instance, lack of motivation and adverse effects resulting from the continuous exposure to computer monitors: No separation of the workplace from the personal. Greater tiredness, degradation of vision, and an increase in the amount of working time. (…) it is a more solitary process, favouring depressive states and anxiety. (…) tiredness and demotivation caused by too much time online.
Inclusion is a contemporary topic, and respondents find antagonistic aspects. On the one hand, ERT connects distant people, and such “wiring” contributes to mitigating inequalities among students. But, on the other hand, this same process may ostracise socioeconomically disadvantaged or info-excluded students. (…) allows mitigating drawbacks that accentuate socioeconomic inequalities. (…) the need for adequate equipment, which is not available to all, further harms students from less favoured socioeconomic backgrounds.
Finally, because lecturers and students work from home, keeping physical distance between them, a significant number of respondents recognize that ERT was and is effective in contributing to the control of the pandemic, therefore impacting positively public health: The possibility of working with the students without the risk of contagion.
even the more skeptical … I am not seen one [advantage] except reduce the viral transmission.
Concerns
Table 3 condenses the themes, subthemes, and aspects that emerged from the open-ended question Concerns. It encompasses four themes: the learning process, self-regulation and compromising the future, resources, the assessment process, and well-being. Notwithstanding that all respondents are lecturers, the themes are considered under two different viewpoints: on the one side, the “offer,” which means the lecturers and higher education institutions, and on the other side, the “demand,” which means the students.
Themes, Subthemes, and Aspects Emerged From the Data Analysis in the Open-Ended Question Concerns.
Learning Process, Self-Regulation, and Compromising the Future
Concerning the theme learning process, self-regulation, and compromising the future, and the perspective on the students, the respondents manifest concerns in that ERT may stunt key transferable competencies like reflection, interpretive and argumentative abilities, or teamwork, critical thinking, and resilience: My biggest concern is that, in distance learning, students do not acquire the necessary competencies and that the job market requires they have. (…) the information needed for one’s education is not only of scientific nature; it integrates other components/dimensions that distance learning does not enable, quite the opposite, it hinders.
Students who have the most significant difficulties in the learning process are another concern. Within this group, the lecturers find students lacking autonomy, those that depend on external stimuli or are more introverted, poor competent/efficient in the study, or just students with weak attendance or who cope badly with schedules and deadlines. In general, the students’ demotivation worries the lecturers, which may manifest in different forms as laxness, lack of commitment and involvement. The dehumanization of the learning process is another apprehension. ERT may transform the student into a hermit, lacking social interactions, which is found to be an essential part of the learning process. Loneliness may not help knowledge acquisition. Finally, all these issues, in concert, may contribute to compromising the students’ future. (…) compromise the future; their own and that of others (the society in general).
Concerning the perspective on the lecturers and HEI side, in general, the respondents extensively (almost one third) cast doubts on the effectiveness of the learning process, that is, they feel that ERT is weak in means to ensure knowledge apprehension and consolidation: (…) on the student side, I feel that the subject matter is relatively little internalized and understood. (…) that learning is not well internalized and solidified.
Practical classes and their unsuitability in online platforms pair with the previous subtheme in what concerns mentions in the answers. Such aspect is expressly addressed in health courses and internships in authentic contexts. Class dynamics is another source of worry for lecturers. It has diverse aspects: communicational (in)competencies to expose subjects in the digital, impaired communication with digital natives, ignorance of strategies in order to maintain the students attention, total lack of feedback, and, as a corollary, lecturers confess that they are missing the clarification of doubts, points-of-view, and other weaknesses: (…) the lack of immediate feedback about the matters taught does not allow for gauging its understanding by the student, which is worsened by the lack of visual contact. Distance learning does not enable putting doubts or different points of view.
Resources
The theme resources encompass several dimensions like technological or financial. One aspect highlighted on the student’s angle is that ERT may exacerbate asymmetries among them, that is, the socioeconomic differences may amplify existing chasms. This environment is also a concern for students who live alone, in a student community, or within the family. It may be connected to the previous aspect, socioeconomic asymmetries, and it is related to access to suitable internet and convenient physical space where the student may work sheltered from interferences: Absence of the necessary conditions to participate in distance learning (e.g., lack of space, noise, interference of younger siblings).
Quitting is another point raised by the participants in the study. It is due to the accumulation of barriers raised by ERT and that fall on the students’ shoulders. ERT may not be convincing in shorting gaps, after all, and distance may prompt quitting: I believe that the probability of demotivation, and the consequent abandonment, is high.
Concerns with technological hurdles come from both sides, that is, lecturers, HEIs, and students, and encompass several aspects. The excessive dependence on hardware and digital devices, the internet, the lack of mastery in such realms, being forced to interact with software fears lecturers, and they imagine that students suffer from similar concerns.
Assessment Process
The assessment process is a recurrent reference among the lecturers’ concerns. Students’ assessment concentrates on specific concerns, from the difficulty in following and validating their progress in digital platforms or the need to seek new methods more compatible with distance learning and the online, to the apprehension that the lecturers’ exhibit with an increased possibility of fraud, which may ruin the evaluation reliability. Some lecturers realize that, on the other side, students perform much better in digital environments than them, and that paves the way to a feeling of insecurity. In parallel, the answers to this question also reveal a flavor of laid-back process, benevolent/easier due to the pandemic circumstances, and less rigor: (…) lack of rigour on the assessments promotes mediocrity (more students achieve success due to lessening the thresholds).
Well-Being
Focusing on the lecturer, well-being is a final theme referred to in the answers when the participants expressed the concerns they expected. The psychological aspects occupy a prominent place within this theme, figuring possible problems on the mental health, the risk of emotional distress and depression. Even agoraphobia was mentioned (extreme anxiety in situations where the person perceives their environment as unsafe and no easy way to escape), mainly due to excessive time facing computers. The risk of failing life parting, between home, the haven, where one may have his/her life sheltered from external aggressions or intrusions, and the “office,” which, much of the time, is physically inside the home, is also voiced: (…) It is very important (…) and separates life at university/professional from the domestic/personal, in terms of space and social interaction.
Solutions
The participants were also asked for possible solutions that might mitigate the disadvantages, soothe the concerns, and envisage some troubleshooting for the challenges foresaw. Table 4 summarizes the themes, subthemes, and aspects that respect the open-ended question Solutions. It encompasses three themes: pedagogy and innovation, normative framework, and digital literacy.
Themes, Subthemes, and Aspects Emerged From the Data Analysis in the Open-Ended Question Solutions.
Before addressing the proper solutions, it must be stressed that a large proportion of the respondents (66 out 344) state that ERT is acceptable during pandemic effects persistence only. ERT role is to mitigate There are no possible solutions. Our teaching model was not projected taking into account distance learning.
Pedagogy and Innovation
According to the respondents, pedagogy and innovation are the most extensive corpus of solutions and come first as sources of answers for ERT’s drawbacks. The theme has several facets, but innovative active pedagogies emerge at the forefront. The lecturers recognize that ERT classes require different pedagogic approaches from the traditional face-to-face teaching: class dynamics must be improved, interactive tools (like chats or forums) must have a daily role, give space to creativity, and intensify the doing. Therefore, there is an investment to be made on active pedagogies, so ERT increases its effectiveness: Find different teaching-learning strategies, active, which may captivate the students. Implement different strategies and dynamics in the class: video-based presentations, build portfolios, handmade work presentations, adapting the teaching method according to the targets.
In the same vein, group working is suggested to solve much of the ERT problems as soon as digital platforms make groups easy and practical. But the implementation of active pedagogies and group working need lecturers, and they must increase their involvement. Increasing the involvement is suggested to pass by investing in the lecturing of the subject matters, greater openness to answer to the student’s difficulties, and, referred many times, include accompanying advisory sessions, ideally individual, that is, more time with the students: Total availability of the lecturer. During the classes and outside classes. It is not acceptable that the lecturer’s full availability at this education level in distance learning does not exist. At several levels: technological, use of assorted contact means depending on the student needs and his/her circumstances.
Online-based courses require specific “for-digital” study materials. Lecturers should adapt the materials to the new media, or even do new ones, from scratch, benefiting from the unique abilities turned possible by technology. Video figures among many answers. But this kind of solution should not rest on the lecturers’ shoulders only. HEIs also have a role in the solutions, for example, deploying free digital bibliographic materials. Nevertheless, HEIs’ role must also be extended into facilitating access to technology, and the respondents extensively pointed out this aspect. Facilitate access to technology means that HEIs must assure that everyone, lecturers and students, have the necessary technological devices at hand: Higher education institutions must guarantee that lecturers and students have access to all means necessary for the classes in the present conditions [ERT].
and internet access, even internalizing the extra costs when HEIs are not able to implement the service: Ensure internet quality; free service or paid by the institution; informatic devices deployed by the institution or contribute to the expenses at least.
and HEIs should accept the role of mediators with banks, much like central purchasing, all benefiting from the scale leveraged: Too many students do not have individual desktops or laptops, for what one would think in reduced interest bank loans negotiated by [name of the HEI removed] and the bank issuing the student card.
Improve access to technology concerns not only hardware and internet but also the support of experts, including information technology (IT) technicians and lecturers that bring the specificities of each field: Equip higher education institutions with the material resources (informatic gear and other kinds of support to the educational activities) and human resources (create teams of lecturers from diverse fields that include technicians pertaining to these areas [ITs]—lecturers are not experts of everything!).
The last subtheme of pedagogy and innovation, adapt assessment, encompasses some clues to work around this issue, which was pointed out as a severe concern previously. The respondents do not propose innovative solutions. For instance, one recurrent suggestion is to assess and examine when the Do the assessment tests in face-to-face sessions only.
Instead, others, although recognizing that assessment in distance learning is a problem because of fraud, suggest increasing and distributing the number of assessment points along the semester and that the evaluations are orals, to mitigate: Adopt more continuous assessments throughout the semester. Drastically reduce the number of students per class because serious assessment is possible in oral examinations only.
Normative Framework
The theme normative framework has a variety of approaches. Firstly, HEIs must put effort into adapting and optimizing the structure of the classes. The design of the classes is related to rethinking the amount of time dedicated to synchronous classes, the duration of online classes, the number of classes per week pertaining to a specific course, introducing asynchronous classes, and the size of the classes. It is factual that synchronous class workload was calculated for the traditional face-to-face system, and the transition to ERT received this inheritance. After months of experience (almost one entire semester), lecturers realize that online classes, particularly ERT classes, run at a higher pace because there are fewer interruptions. Therefore, maintaining the same paradigm, the lecturers concluded that there is no need for the same amount of time for synchronous classes. Also, because online synchronous classes run at a higher pace, they are more tiring and therefore, their duration should be shortened. To maintain the same class workload, the number of classes per week could be increased. In parallel, the lecturers realize that expositive classes could be delivered asynchronously, that is, ERT may opened the door for asynchronous classes. Still in the structure of the classes, a recurring demand of the respondents is to reduce the number of students per class, some of them even declaring its limits to 10 to 15 students. Finally, within the subtheme structure of the classes, there are no proposed solutions for the inadequacy of practical classes in ERT. A significant number of answers (46) suggest that there should be complementarity between distance learning and face-to-face classes, the latter specialized in practical learning, that is, apply the structure of blended learning: I think that alternating between face-to-face and distance learning systems would improve both types of classes. Associate distance classes with face-to-face, with a smaller number of students.
Secondly, an aspect often raised is that students must keep their webcams switched on: Try to replicate, in real-time at [name of the digital platform removed], the face-to-face dynamics, asking students to switch on their webcams and keep them that way during the class.
and some answers emphasize that this norm should be enforced in regulations: The use of webcams should be mandatory.
Thirdly, there are calls for HEIs (internal) regulations and governmental (external) too. Such regulations would have to enforce distance learning organics both internally and in all HEIs (externally). Some respondents rely on the HEIs’ government bodies: Better and anticipated decision-making by the [HEIs’] government bodies that facilitates the lecturers’ organizing and preparation of study materials and assessment methods.
Digital Literacy
The third theme in the solutions for ERT concerns digital literacy. The watchword voiced here is training. Training, both for the academic staff and students too. Students are not forgotten, and the respondents noticed that they also need training specific for learning in the online realms. The matters to be covered in lecturers’ training focus on digital-based teaching platforms, digital tools, modes of online education, and appropriate pedagogic methods for distance learning environments: Besides, it is important to attend some courses about the specificity of distance learning (I am not referring to technological competencies but pedagogic tools and didactics specific to this educational mode). Digital inclusion, training.
Discussion
The study reflects lecturers’ perspectives about the ERT process developed throughout the second semester 2020/2021, that is, roughly from middle March until July. This period allowed to analyze lecturers’ opinions on the challenges arising from the whole ERT process, including planning, class development, and assessment. A viewpoint that one must consider when interpreting the results is the mixed origins of the lecturers, whether working at universities or polytechnics, public or private, and coming from assorted areas.
A noteworthy fact is the vast number of participants that responded to the open-ended questions. A few answering fields were kept empty (36 out of 344 × 4 = 1,376 answering fields), suggesting that situation was impactful, and people were eager to share their experiences. Thus, it is not surprising that, when lecturers were asked about the advantages and disadvantages of their recent experience with ERT, they focused primarily on the traditional lecturing-learning paradigm. Nonetheless, the detail put in the answers is remarkable, which contributed significantly to the richness of this study, as shown in the extensive tables that sum up the respondents’ contributions.
A global appraisal of the findings reveals that the pedagogic strategies, the management of the class, here included the communicative issues with students, coupled with their learning performance, figure pervasively among the advantages, disadvantages, concerns, and the solutions as well. Yet, lecturers also recognize that ERT impacted themselves, individually, and over society and the environment.
On the delivery side, that is, lecturing, one does not find extensive concerns about ERT, apart from the technological hurdles, practical classes, class dynamics, and the assessment process, although these are seen as disadvantages. But, conversely, one finds many solutions to be implemented through pedagogy and innovation (cf. Table 4). In fact, pedagogy and innovation represent a significant part of the proposed solutions. Thus, it seems that lecturers recognize the drawbacks of online teaching and that some concern them. Still, they also envisage solutions that may mitigate or solve the problems without forgetting their merits.
One aspect that seems to have no solutions, yet at least, is practical classes. Online education in general, and ERT in particular, have advantages in theoretical and theoretical-practical classes. Not in pure practical classes, more specifically in laboratory classes (cf. Tables 2 and 3). One must consider the origin of the respondents: around 40% lecture in health and engineering faculties, the former containing extensive clinical practice and the latter with extensive laboratory classes. Such origins may influence the recurrent critics stressing the inadequacy of ERT and online education for practical classes. Maybe practical classes in other disciplines may be more or less virtualized as in languages, accounting, business, or informatics. In other areas, like sports, maybe it is challenging, ineffective, or even impossible.
Returning to a global discussion of the results, there are contradictory contributions, such as online and distance. On the one hand, educational technologies, mainly those concerning online communication, may bring students away into the classroom. Such facility helps to mitigate long commutes, bad weather conditions, illness, student-worker hindrances, and so on. On the other hand, these same technologies take the student away from the classroom, as s/he may not resist the temptation to attend the class benefitting from the home comfort, contributing to a certain dehumanization of the learning process. It is advantageous for managing big classes (admittedly because side talks among students or other kinds of distractors do not exist in online classes or their effects do not disturb the course of the class), and it also makes it difficult to manage big classes (admittedly because the lecturer has the screen cluttered with students’ photos and names and/or is unable to manage an avalanche of queries during a q&a period). It helps to improve students’ key competences (stressing the digital ones) and also precludes students from learning and developing key competences (e.g., teamwork or dialogue). Help keep students focused on the online class, and students do not participate in the class and it is difficult to motivate them for such. Ostracises socioeconomic disadvantaged students and mitigates inequalities among students. The same situation is seen as advantageous and disadvantageous throughout several contributions of the lecturers. One may recognize that one situation can encompass positive and negative arguments. However, such distinctions may derive from the lecturer’s propensity towards using technology in education and his/her comfort and savvy with it, and such predisposition may be scientific area related. It sounds normal that, in general, STEM lecturers show comfort with technology, while in arts, humanities, or law areas, lecturers tend to show less ease or even some resistance and antagonism. Although our data does include extensive contributions from the latter group, a recurrent conclusion among the respondents is that training, either improving general digital literacy or focused on digital-based innovative pedagogies, would be strongly beneficial (cf. Table 4). As a matter of fact, the answers of some lecturers reveal unfamiliarity, or even ignorance, of functionalities that educational or communication software already deploys. For instance, one of the disadvantages recurrently pointed in online teaching is working in groups (cf. Table 2 and the end of section 3.1.3). However, either the breakout rooms in Zoom, or channels in Microsoft Teams, may be used for group work, just bringing two examples, containing enhancing functionalities, the latter deploying file-sharing or collaborative work in documents, besides chatting. Another functionality admittedly ignored, and pertinent to students’ assessment, is the e-proctoring software, which turns possible the traditional written quizzes/exams, now with students distant, which could contribute to overcoming the sensation of unreliability and existence of fraud. Hence, adequate training, targeting digital-based education, could contribute to solving an important part of the concerns raised by the lecturers. Yet, some other lecturers have already pointed out those same aptitudes of digital-based education and sometimes even consider them as advantages/solutions.
However, the need for training does not circumscribe to the lecturers’ side. Students also need to adapt to the digital/online environments indeed, with a particular focus on studying and learning in such context, as per the respondents’ contributions. They identify difficulties on the student side as lack of participation in the synchronous classes, deficits in self-organization and imposing a working agenda, loss of reference and objectives, self-isolation derived from the absence of extra-the-class social flirting, which could lead to giving up on university. Students must cope with these hurdles, which may be not new, but which are enhanced in online environments, and training may help prepare them to deal with them. One could admit that Generation Z, most of the HEI’s students, at least at the bachelor level, would already be acquainted and comfortable with online life. However, even an immediate approach reveals that the participation of the digital in their lives is limited to social network chatting, which is a pale use of the available technologies and is limited in what concerns the improvement of the learning process. Hence, students also need training for online-based learning, for example, a stressed aspect, how to behave in front of a switched-on webcam and how to participate in online meetings, which is already a requirement in real-world organizations daily life like companies.
Considering the solutions suggested in the answers to the open-ended questionnaire (cf. Table 4), expecting that they might contribute to mitigate, or even solve, concerns or disadvantages raised by the lecturers (cf. Tables 2 and 3), Figure 1 depicts key aspects collected among the advantages (cf. Table 1) that could constitute the main girder of a training program. Most of the themes apply to lecturers, but students must also learn to effectively study in digital-based educational environments.
Central themes for a training program, either for lecturers or students, collected from the open-ended questionnaire contributions.
Four topics make the Digital Competences theme. Firstly, there are the General Digital Competences that both lecturers and students must acquire. That is the purpose of the European Union’s Digital Competence Framework 2.0 (DigComp; https://ec.europa.eu/jrc/en/digcomp), which designates 21 competences in five areas (information and data literacy, communication and collaboration, digital content creation, safety, and problem-solving) that every citizen must be comfortable with. HEIs, mainly those located in the European Union, should ensure that their academic staff master all of them and certify that every student does not leave the university or polytechnic school without being digitally competent. Secondly, the Well-Being in Digital Settings. Working for hours sitting in front of a monitor requires adaptations that mitigate the problems identified by the participants. Ergonomic issues are not the most mentioned, but psychological ones. Of course, a monitor flickering may cause damages. But the problems highlighted refer to the impacts on life organization (one may jump from the bed to the computer, sometimes still wearing the pyjama, and then has lunch and returns to the computer, assuming that the lunch is not in front of the computer), and the sense of self-isolation, abandonment and dehumanizing that may slip into demotivation. Finally, there are Transgenerational Digital-Based Communication and Webcam Behavior. Webcam Behavior has a particular highlight due to the recurrent reference among the answers. Maybe General Digital Competences can minimize the difficulties in the communication between lecturers and students mediated by the digital. Students usually communicate easily among them through the digital, for instance, social networking. Still, there is evidence that they choose different social networks so to avoid exposure and cross-talking with the elderly. Such chasm disturbs the teaching-learning process, as it also involves different generations, and training should focus on eliminating the barriers.
Six topics make the Digital-Based Innovative Pedagogies. The first subtopic is Mastering Digital Education Software. Lecturers must master the software, acknowledging that many students master the digital realms, exercising dominance over the lecturer and risk the knowledge transfer process. Secondly, and heavily stressed among the contributions in the open-ended questionnaire, Active Pedagogies for Digital Education Environments. Expositive classes, where the lecturer orally explains the slide contents, are usual and predictable in higher education. The lecturer controls the session using students’ expressions and manifestations. However, in online classes, students’ expressions and manifestations miss and the lecturer has no feedback to help him/her control de audience. It seems s/he is talking to an anechoic camera. Therefore, if active pedagogies are suggested in everyday classrooms, they seem to be mandatory in synchronous online classes. Thus, the subtopic Active Pedagogies for Digital Education Environments should stress on:
○ class dynamics;
○ manage a big audience;
○ control absence;
○ coordinate students working in groups;
○ monitor students’ progress;
○ motivate students’ participation and spontaneity;
○ recruit students’ attention and concentration.
Close to the previous topic, thirdly, training should deal with Lecturer’s Posture in Digital Environments. This topic has diverse facets, such as visually behaving in front of a camera, voice, prosody, discourse, and mannerisms. Fourthly, designing and producing “For-Digital” Studying Materials. As many lecturers have pointed out, online lecturing opened the opportunity for disparate pedagogic materials, enhancing learning effectiveness. However, for example, the lecturers must now know how to design and produce educational videos, or narrated powerpoints, or screencasts, which requires mastering new hardware and software. Fifthly, another concerning aspect raised by the lecturers is the assessment reliability in online settings. The traditional written quiz/test containing closed or open answers is out of control, and fraud may exist. Besides other ways to assess students and which are suitable in online settings, e-proctoring software and services help make them possible. Therefore, training should focus on both, that is, on alternative ways to assess the students’ technical competences and on e-proctoring. Finally, Transferable Competences (besides the digital). Although the Digital Competence Framework 2.0 (DigComp; cf. footnote 8) focus on digital competences, European Union also has the “Recommendation on Key Competences for Lifelong Learning” (cf. footnote 6), which are “(…) essential to citizens for personal fulfillment, a healthy and sustainable lifestyle, employability, active citizenship and social inclusion.” The eight competences are:
○ literacy;
○ multilingualism;
○ numerical, scientific, and engineering skills;
○ digital and technology-based competences;
○ interpersonal skills, and the ability to adopt new competences;
○ active citizenship;
○ entrepreneurship;
○ cultural awareness and expression.
Although the digital competences figure again, HEIs, mainly those in the European Union, may include the achievement of these competences in their training program for their academic staff and educational programs for their students. Such transferable key competences would help mitigate, or even solve, disadvantages and concerns like managing workload, self-organization, and so on.
Limitations, Conclusions, and Further Research
As is current in qualitative studies, context shapes data acquisition and conclusions. The context is geographically limited as respondents are mostly from the North of Portugal and from governmental-owned HEIs, which may get the tenure in the early stages of their careers. Also, a representative sample was not enforced, for example, ensuring respondents from all the academic fields weighted according to the number of students. All answers were considered and the distribution among academic fields reflects the authors’ origins. In any case, and because this is a qualitative study, no magnitudes were taken into account and just the emergent categories and themes were considered. Finally, data acquisition took place during the pandemic and after a full semester of ERT, that is, during a “hot” period. Such context may have exacerbated some participations. Because respondents’ answers were collected in an online form, non-verbal information is missing.
The emergency remote teaching (ERT) that numerous HEIs had to introduce in the second semester of 2020/2021 (roughly from middle March until the end of July) forced many lecturers into the world of online education, most of them with minimal training for that, or even with no training at all. However, some lecturers already had training and were using new pedagogic strategies, like the flipped classroom or working in areas where there is empathy on technology, especially digital technology. The open-ended questionnaire gathers contributions from them all, which is a unique aspect of this study. There are those that express a nihilist evaluation of ERT and online education, those that saw disadvantages and raised concerns about their recent experience, and also those that perceived an opportunity in ERT, so to improve the teaching-learning effectiveness coupled with an occasion to introduce achieving key transferable competences in addition to the technical ones. Lecturers participated extensively with many contributions. These contributions converged in themes.
Although the online realm may not conveniently replicate some physical world phenomena, for example, clinical practice or engineering laboratories, the lecturers claim for generalized training to overcome the disadvantages and concerns on ERT and online education. In fact, perusing the advantages, one may find the solutions for many the risen problems, that is, the data suggests that a fraction of the lecturers found disadvantages or rose concerns because they are unaware of the possibilities allowed by digital-based education. Hence, training may be the most relevant solution, training for lecturers and also training for students, as this new learning environment requires adaptation.
The study revealed the two main girders of a putative training program, 1) on Digital Competences, and 2) on Digital-Based Innovative Pedagogies (cf. Figure 1). The former unfolds in General Digital Competences, Well-Being in Digital Settings, Webcam Behavior, and Transgenerational Digital-Based Communication. The latter unfolds in Mastering Digital Education Software, Active Pedagogies for Digital Environments, Lecturer Posture in Digital Environments, “For-Digital” Studying Materials, e-Proctoring, and Transferable Competences (besides the digital).
Lecturers and students equipped with such training may be better prepared for another eventual emergency, or for real real-life, as many organizations already have such requirements today. As some respondents echoed in the open-ended questionnaire, the solution may be blended-learning and innovative pedagogies.
The exploratory nature of the present study aimed to make emerge the constructs pertinent for ERT derived from the lecturers’ experience and perspective. The conclusions call for a confirmatory study and test which are pertinent and effective, in fact. Figure 1 summarizes the main themes that should be addressed during training in the respondents’ perspective. It is time now to test if accordingly training interventions may make the lecturers more able to deal with the exigencies of 21st-century students’ teaching.
Footnotes
Acknowledgements
We would like to thank the institutions and professional organizations that promoted the study on their internet platforms (Faculty of Medicine of University of Porto, FMUP and the Centre for Health Technology and Services Research, CINTESIS), and all the lecturers who participated in this study.
Author Statement
José Paulo Marques dos Santos: Conceptualization, Methodology, Formal analysis, Data curation, Writing—original draft, Writing—Review & Editing; Carla Serrão: Conceptualization, Methodology, Project administration, Supervision, Writing—original Draft, Writing—Review & Editing; Teresa Sofia Amorim Lopes: Formal Analysis, Data curation; Ana Rita Rodrigues: Writing—original Draft, Writing—Review; Silvia Marina: Writing—Review; Andreia Teixeira: Formal Analysis, Writing—Review & Editing; Luísa Castro: Formal Analysis. Writing—Review & Editing; Ivone Duarte: Conceptualization, Methodology, Investigation, Supervision, Project Administration, Funding Acquisition.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by FCT—Fundação para a Ciência e a Tecnologia, I.P., within CINTESIS—Center for Health Technology and Services Research [grant UIDB/ 4255/2020], and inED—Centre for Research and Innovation in Education [grant UIDB/05198/2020]. Teresa Sofia Amorim Lopes is supported by FCT—Fundação para a Ciência e a Tecnologia, I.P. through the PhD grant SFRH/BD/135668/2018. Ana Rita Rodrigues is supported by FCT—Fundação para a Ciência e a Tecnologia, I.P. through the PhD grant SFRH/BD/145253/2019.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.