Sage Journals: Discover world-class research

Abstract

The aim of the current study was to develop an application for mobile devices that can improve the organization and documentation of clinical teaching in dental schools. The app should include a calendar to organize treatments, an overview of all treatments including the treatment stages, a digital waiting-list for the teacher’s assessment, and a standardized recording of all assessment results. Students and members of a Medical Faculty and a Faculty of Computer Science and Business Information Systems jointly developed the app. Twenty-three students tested the usability after 4 weeks. Next, the new app was tried and tested by n = 79 forth-year students who used it for 2 months during their clinical course. Fifth-year students of the same clinical course who did not use the app constituted the control group. Afterwards, the app was standard implemented in both integrated clinical courses and evaluated after 1 year by the same students both in winter and summer semester. After usability test, the majority of students (81%, n = 18) said they would recommend the app and gave it a mean overall score of 2.0 ± 0.6. Next, course organization and assessment was evaluated by n = 75 fourth-year students (95%) who used the app and by n = 40 control students (74%) who did not use the app. Students who used the app reported significantly better organization of treatment (p = .002), documentation of treatment (p = .001), waiting for the teacher’s assessment (p = .038), overview of treatment sessions (p = .040), and transparency of grading and assessment (p = .012). After 12 months, the now fifth-year students still gave very good ratings for the organization of the treatment and the transparency of the grading in both evaluations, so that no significant changes were detectable (organization p = .959 and the rating p = .751). The newly created app improves the organization and evaluation of clinical education and even after 1 year of service, the students still perceive the consistently high advantages of digitization.

Keywords

Application mobile devices clinical education pre-gradual dental students

Introduction

In an era where digital technology pervades every aspect of our lives, its integration into the educational sector, especially in specialised fields such as dental education, has become crucial. This paper attempts to navigate through this digital terrain, presenting the development, implementation and evaluation of a mobile application specifically designed for dental education.

Emergence of digital learning across disciplines

Digital learning has reshaped educational paradigms across disciplines, introducing innovative teaching and learning methods. Digital platforms and tools have not only made education more accessible, but have also required changes in how educators approach teaching and how students engage with learning. While (Alamri et al., 2021) highlight the increased flexibility and personalisation of learning experiences for students, (Bates, 2015) note that this shift towards digital media requires educators to continually adapt and innovate. Digital teaching elements can help students adapt to curricular structures and promote individual learning (Divaris et al., 2008) and can be useful for both students and teachers. Nevertheless, this digital revolution also brings challenges, including issues of engagement in virtual environments (Rajabalee et al., 2020), redesigning of course materials, and an good digital infrastructure as resources, training and support (Anand et al., 2014).

Digitalization in dentistry and dental education

Digital technologies are reshaping dental education, mirroring their transformative impact on clinical dentistry. Digital systems are used for diagnostics and dental decision-making. For example, efforts are being made to establish artificial intelligence (AI) in various patient care systems. These efforts promote individualized care and are expected to improve long-term treatment conditions and reduce costs (Schwendicke et al., 2020). Students should be exposed to digitalization early in their dentistry studies, so that they can acquire a basic understanding of these processes ready for clinical application later on. Digital systems are also used in medicine and dentistry to train psychomotor skills. A range of virtual reality (VR) systems are being used to support clinical training. These include 3D-supported headsets that simulate virtual patients and systems (such as digitally enhanced phantom heads) which enable haptic training, fine motor preparation skills, ergonomically optimized work, and automated evaluation of preparations (Tang et al., 2021; Towers et al., 2019). Classic dental teaching formats can also be supported digitally. 3D printing can produce dimensionally accurate models based on intraoral scans (Rungrojwittayakul et al., 2020) and it is even possible to produce individual teeth for training purposes, for example, in endodontics (Shah and Chong, 2018).

Mobile learning and digital devices

As students already have good digital skills (Khatoon et al., 2019), they should have no problems using digital teaching systems. Many students already use mobile devices (e.g., tablets) in class to structure or annotate their teaching material (Anand et al., 2014). The advantage of these devices is that learning material is constantly available during the clinical course (Pyorala et al., 2019) and can be used in a self-paced manner. Today´s generation of students are digital natives, and the use of these devices is integrated fully into their daily lives. Therefore, the majority of students have a positive attitude towards mobile learning (Wallace et al., 2012). As a consequence dental students wish and expect the integration of multimedia systems as part of their learning experience in dental studies (Mayer, 2014). The Students benefit from the use of mobile devices as an online source of information (Tews et al., 2011), appreciate having digital course materials always at their fingertips, and express that information technology in the classroom enhances their learning (Law et al., 2018). A number of medical schools have been successful in the integration of mobile devices into their curriculum and learning environment (Leydon and Schwartz, 2020). The use of tablet computers, in particular, leads to an improvement in the search for information and in the time management of students (Patel and Burke-Gaffney, 2018). Numerous applications have been established to transfer theoretical knowledge (e.g., digital versions of large reference works, dictionaries, practice exams) in dental teaching. In clinical settings, for example, there are applications that support the management of dental trauma treatment (Al-Musawi et al., 2017; Khehra et al., 2021), X-ray diagnostics of endodontic complications (De Oliveira et al., 2019), oral hygiene at home (Scheerman et al., 2020; Underwood et al., 2015) and the education of smokers about oral cancer (Nayak et al., 2018). It has been shown that using mobile devices as a source of information in daily work can be helpful, especially for young students who are developing their skills (Bullock et al., 2015; Webb et al., 2016). Digital technologies expand and enhance traditional teaching methods and may improve teaching, learning, performance evaluation, and documentation while increasing student satisfaction and teaching performance (Morrow et al., 2014). Apps for mobile devices are being developed to support students and clinicians in clinical decision making and to provide students with timely feedback at work (Patel and Burke-Gaffney, 2018; Payne et al., 2012).

Challenges and organization of the integrated clinical courses of dentistry (Background Information)

Despite all efforts towards digitalization, teaching clinical skills within the curriculum is one of the greatest challenges in dental education. This process involves not only the student, but also the clinical course assistant and the patient being treated as well. A number of factors make this part of dental education one of the most difficult: communication within the student treatment team, the communication skills of the student towards the patient, the didactic skills of the clinical course assistant, and the theoretical and practical preparation of the student for patient treatment (McGleenon and Morison, 2021). With regard to organizational structures, dentistry students have to organize appointments and treatment plans, and prepare instruments and materials required for treatment. They also have to document file entries and record their course performance. Numerous studies have reported high stress levels among dentistry students in response to exam situations and the demands of practical pre-clinical training and treating patients (Alzahem et al., 2011; Elani et al., 2014). Research has shown that the transition from pre-clinical to clinical studies caused a subjective increase in stress levels among students (Frese et al., 2018). The situation of having to treat patients under the supervision of course assistants from the fourth year onwards is a particular challenge of the course. This is unique compared to all other university degree courses and also represents a significantly higher psycho-social burden compared to studying human medicine. These decisive factors are regularly reflected in internal evaluations, which praise the high quality of the content and scope of dental training. However, they also highlight the aspects that cause stress for dental students. As the work processes in the clinical courses are primarily patient-centred, students do not always receive immediate feedback, as the clinical course assistants coordinate and supervise six patients and 12 students at the same time. This is described by students as a feeling of insecurity. Together with the lack of clinical experience, this can lead to feelings of inadequacy. To improve this situation, course assistants need to be supported by a tool that allows for more efficient prioritization of treatments, systematic assessment, documentation and communication of student performance.

According to the requirements of the new licensing regulations for dentistry (ZApprO according to German law), interdisciplinary clinical courses must be introduced at all dental faculties in Germany by October 2021. This means that several disciplines in dentistry are obliged to host integrated clinical courses for dental students what represents a major organizational and didactic challenge for most universities. For example, different doctrines and opinions must be coordinated across departments and implemented reliably in everyday clinical practice despite staff changes. For students, the complexity of treatment processes is also on the increase. Patient-centred care now characterises the course activities instead of clearly delineated treatment tasks that have to be completed within one semester at the latest. As a result, patients are often cared for over the course of a whole year, and all the treatment needs have to be recorded, planned and carried out by the students in an interdisciplinary manner. This means coordinating treatments, managing waiting times and integrating all individual treatments into a sensible plan. More and more, and especially with the change of regulations due to the ZApprO, treatment appointments must be planned more flexibly. While our faculty has already taken significant steps to address these common sources of stress with the revision of special catalogues, a communication programme for dealing with difficult patients, and coaching to strengthen personal resilience and academic work style, the focus is now on organisational overload.

Aim and scope of the paper

To address these educational challenges, particularly in the realm of course organisation, grading and assessment we developed a mobile application in collaboration with a neighbouring Faculty of Computer Science and Business Information Systems. The aim of this paper is to present the development and implementation of this mobile application, evaluating its efficacy in improving the organisation and assessment processes in dental education. The null hypothesis was that a mobile application would not improve organization and assessment of clinical teaching in dentistry.

By exploring the broader implications of this digital integration, the paper aims to offer a comprehensive understanding of the potential and challenges of employing digital solutions in higher education.

Materials and methods

This project received ethical approval by the local Medical Ethics Committee of the Medical Faculty (reference number: S-588/2021).

We performed a user and requirements analysis in the dental school at the Medical Faculty to define the organizational needs for an app in the integrated clinical treatment course. The participants were the semester representatives of the 4th and 5th year students, the teaching coordinators, the dean of studies and the course leaders of the respective clinical courses. The results of this analysis defined four basic conditions for the app: (i) a calendar system that provides an overview and organization of treatments and enables real-time synchronous ordering in of patients in the treatment unit, (ii) an overview of all treatments including the different treatment stages, (iii) a digital waiting-list, and (iv) a standardized record and up-to-date summary of all reviews the students have received during their clinical course. A proposal for implement this project as part of their computational science program was applied to a neighbouring University of Applied Sciences. This was a required course for the students of the business informatics, providing real-world experience working with public/private businesses in developing mobile applications.

Functional requirements

Figure 1 shows the main functional requirements (FRs) of the app, which are listed in descending order of priority in the following listing according to the results of the analysis. Some functions are equally accessible to teachers and students, for example, FR2 and FR3, while others are specifically adapted to teachers or students.

(i) FR1 – creation of a treatment session;

(ii) FR2 – personal login and account at the university data center;

(iii) FR3 – a wiki summarizing all treatments administered during the clinical courses; the different treatment steps, that have to be presented by the students to their teacher;

(iv) FR4 – a digital waiting list; the students should send a ticket via their mobile device when they want to present a treatment step. The teacher will be notified of this request via push-message on their own device;

(v) FR5 – an grading and assessment system. The student’s achievements during the integrated course are recorded, for example, as points, scores, grades for each subject.

Figure 1.

Overview of the functional requirements of the app according to the allocation of functions. Some functions were accessible to teachers and students while others were specific to the teachers or students. FR = functional requirement.

Non-functional requirements were also defined. These describe the general conditions that must be fulfilled for the new app to be accepted, including a security concept, compliance with data protection guidelines, the system environment, easy operation, and availability.

Project flow

Students in the sixth semester of the business informatics course at a University of Applied Sciences were asked to develop the app within the scope of a graded student project. First, the seven-member group of students dealt with typical requirement engineering tasks. This initial start of software development tries to optimize the quality of requirement definitions and leads implicitly to a significant reducing of misunderstandings (Bendel, 2018). The result of the first iteration was an offering presentation of the business informatic students to dental staff. They described a first concept of the future application derived by the preceding analysis and introduction of a preliminary schedule. This presentation ended with the start of the first sprint, which is the main entity in the SCRUM development method which was used to develop the app. This state-of-the-art approach in software development allowed expenses to be predicted and risk control to be optimized (Schwaber and Sutherland, 2011). Fixed meetings and reviews between the development team and the responsible course leader at the university every 2 weeks are mandatory in this approach and marked the end of a sprint alongside the start of a new one. In this way, the app requirements outlined in the previous section could be discussed and adjusted as needed during the whole development process. Each meeting consisted of two parts. The functional acceptance and the preview of the new iteration. In the functional acceptance the business informatic student group present their implemented features, fixed bugs or just explain several problems which need to be solved to continue efficiently. The preview of the new iteration contained the goals of the student group in the following sprint. To avoid unnecessary work, every new feature was discussed with the course leaders during this preview based on a wireframe or scribble. All in all, the project flow and development process could be described as customer oriented. It was characterized by the approach of design thinking to focus on the user and his needs and enable a better performance in a foreign field for the development team (Bendel, 2018).

Technical implementation

The mobile app needed to be able to store data and provide a communication channel between the mobile device and the internet. These functions had to be implemented in the background via backend services. To do this, the project team was divided into two groups. One group implemented the app and administered the Web site (frontend) while the other group implemented the backend services. The interfaces between the frontend and backend services were defined by the whole group. This created a distributed system during app development, as illustrated in Figure 2. The app development took two semesters. It is important to note that the application does not store or otherwise process sensitive/personal patient data. To test the app, the medical faculty procured mobile devices (iPad Air 3, Apple, CA, USA) and loaned these devices to the dentistry students.

Figure 2.

Schematic description of app services.

Administration site development and mobile devices

The content of the app such as regularly changing course lists of participants, adjustments in grading and assessment and wiki updates can be uploaded via Excel imports and maintained by a web-based administration system (portal).

Every time the user enters the application, the application automatically seeks and downloads all updates. Students and lecturers were equipped with mobile devices (iPad Air, Apple CA, USA). These were leased from the medical faculty of the University and loaned to the students. We opted for a lease and for the iPads because the core features that are important to us, such as ease of use, recovery and setup, bulk recovery and setup, app purchase and distribution, operating system uniformity, and future viability, are currently only available from Apple, CA, USA (Mittmann and Obst, 2013). The mobile devices were integrated into a Mobile Device Management (MDM) and could be managed centrally by the IT group of the dental school of the faculty. In addition to the newly developed application, numerous other applications available in the app store were installed on the students’ devices to support teaching.

Usability test

The first usable version of the app was placed on mobile devices using a Mobile Device Management (MDM) program. Mobile devices were given to a group of students (n = 23) and a group of teachers (n = 3) to test app usability during the 2020 summer semester of the integrated clinical course. After 4 weeks, a usability test was administered to students to measure quality based on 14 items, each of which was rated according to a five-point scale, where 1 = very good and 5 = poor.

Table 1 illustrates the content of the usability survey. This usability test generated data for optimizing user-friendliness and showed whether the app was likely to be accepted by users and improve the processes in clinical teaching.

Table 1.

Overview of the test used to determine app usability.

Item nr	To what extent do you agree with the following statements?	Response scale (1 = completely agree; 5 = disagree)
1	I Would like to use the app more in future	1 2 3 4 5
2	The app is easy to use	1 2 3 4 5
3	The app is difficult to use (e.g., unnecessary views, important things difficult to reach)	1 2 3 4 5
4	The various functions are well coordinated	1 2 3 4 5
5	The app is inconsistent	1 2 3 4 5
6	I Feel confident using the app	1 2 3 4 5
7	I Think that using the app will give students a fairer assessment	1 2 3 4 5
8	I Think that the app reduces the organizational burden of the course	1 2 3 4 5
	Please score the following	Score (1 = very good; 5 = inadequate)
9	Service	1 2 3 4 5
10	Layout	1 2 3 4 5
11	Clarity of the individual pages	1 2 3 4 5
12	Overall structure	1 2 3 4 5
13	How would you rate the app overall?	1 2 3 4 5
14	Would you recommend the app for other courses?	Yes/No

Mobile application features and use (Students Version)

At the moment there are 4 basic functions available.

Treatment planning (calendar function)

This feature includes real-time synchronised patient scheduling for students assigned to a treatment unit.According to EU data regulations and for privacy reasons, no patient data is used. Instead, the student selects the upcoming treatment procedure from a drop-down menu and schedules a treatment appointment. As the appointments are often organised by the students themselves outside of course hours, this makes it easy for them to co-ordinate with their student treatment partner. At the same time, the student is booked into a specific treatment unit. This avoids duplication of use.

Wiki

The wiki is a collection of standardised procedures for the dental treatment of patients in the disciplines of conservative and prosthetic dentistry. On the one hand, this is a measure of reference and quality assurance for the students who are treating the patients. On the other hand, the assisting treatment partners can inform themselves in advance about the necessary materials and instruments without having to make complicated arrangements and have them ready in time. This is particularly advantageous when less experienced students are assisting or when the treatment partners do not know each other personally.

Digital Waiting List

The digital waiting list is activated at the start of the dental treatment session. It contains all standard treatment steps, which can be adapted to individual patient situations by course assistants. The course assistant can be notified at the touch of a button when a treatment step has been completed or when further enquiries are required. This eliminates additional travel for students and, where applicable, disadvantages for more geographically distant treatment units.

In the event of an emergency, it is also possible to send an alarm message to all course assistants. This is intended to help reduce the risk of confusion in a situation such as this.

Assessment

In the assessment menu, students can view their current score in the different parts of the clinical course (conservative and prosthetic dentistry), as well as the treatment time used so far, which in particular allows for better coordination with the treatment partner or the course assistant to avoid unfair situations.

Mobile application features and use (Course Supervisor Version)

There are currently 4 basic functions.

Overview of all treatments in the treatment units

Clinical course assistants have a complete overview of all scheduled treatments in all student treatment units. A 'Favourites’ function allows the course assistant to select only those students who need to be supervised at that particular time. The overview takes place in real time, as soon as the students have made their appointments.

Wiki

The wiki is identical to the corresponding student version of the application. This ensures that students and course assistants are working to the same standard. This is particularly advantageous if course assistants are only in charge of the student course on a substitute basis.

Digital waiting list

The digital waiting list allows the clinical course assistant to priorize the student’s session attendance. As soon as a student has successfully completed a treatment step, the course assistant will receive a ticket. Students can only continue with treatment once they have been released, usually after a consultation in the treatment box. In addition to the organization of the working day, this is also a formal hurdle that prevents students from continuing ‘on their own'.

Assessment

As soon as the students have completed a dental treatment session in conservative or prosthetic dentistry, an assessment dialogue automatically opens in which the quality of the outcome and the process must be evaluated with a score or grade. This ensures that the assessment takes place promptly and with a fresh impression of the course events. The feedback function to the student allows for timely queries if necessary, so that potential conflicts can be avoided.

Evaluation

In the 2020/21 winter semester, mobile devices with the pre-installed app were handed out to six teachers and 79 forth-year students of the integrated dentistry clinical course. The students and the teachers were first provided with instructions which walked them through how to use the mobile app. After using the app for 2 months, the participants evaluated the organization of their course and documentation of their performance in a 17-item survey. This determined whether the app adds value to the integrated clinical course in dentistry. As a comparison group, 54 fifth-year dentistry students of the clinical course, who did not have access to the app, completed the same survey (Table 2).

Table 2.

Survey for evaluating clinical course organization and performance documentation for both forth-year and fifth-year students.

Item nr	To what extent do you agree with the following statements?	Response scale (1 = completely agree; 5 = disagree)
1	The organization of patient treatment during the course was very time-consuming for me	1 2 3 4 5
2	Scheduling and coordinating patient treatments during the course was easy for me	1 2 3 4 5
3	The individual treatment steps were given transparently to me	1 2 3 4 5
4	I Could find important information before or during treatment	1 2 3 4 5
5	During the course it was clearly regulated who was in which treatment box each day	1 2 3 4 5
6	Patient appointments were clearly documented during the course	1 2 3 4 5
7	Dates for individual boxes were clearly documented during the course	1 2 3 4 5
8	I Had a good overview of my scheduled patient treatments during the course	1 2 3 4 5
9	Box allocation was well coordinated among the students this semester	1 2 3 4 5
10	The waiting time for presenting a treatment step was appropriate	1 2 3 4 5
11	The course assistants adhered to the sequence in which a treatment step was presented	1 2 3 4 5
12	It was clear to me which treatment steps I had to show to the course assistant	1 2 3 4 5
13	In an emergency I got immediate support from the course assistants	1 2 3 4 5
14	I Had a continuous overview of my daily performance during the course	1 2 3 4 5
15	I Had a continuous overview of my number of treatment days	1 2 3 4 5
16	I Had a continuous overview of my overall performance in the course	1 2 3 4 5
17	The course services were documented transparently and in a standardized manner by the course assistants	1 2 3 4 5

Evaluation after 12 months of standard implementation

The mobile app was standard implemented in integrated clinical courses starting in the winter semester 2021/2022. Within the regular evaluation process two additional items (Table 3) were added as part of the central evaluation. Evaluation was done by fifth-year students two times, in the winter semester (n = 76) 2021/2022 and in the summer semester 2022 (n = 75).

Table 3.

Overview of the items used in winter semester 2021/2022 and summer semester 2022 in the integrated clinical course 2 (IC 2, fifth year Students) central evaluation.

Item nr	To what extent do you agree with the following statements?	Response scale (1 = completely agree; 5 = disagree)
1	Using the app to organize the integrated clinical course 2 (IC 2) is helpful	1 2 3 4 5
2	The use of an app to assess and view my course performance on a daily basis should be permanently established in the course	1 2 3 4 5

Statistical analysis

Data were assessed and analyzed using SPSS (IBM SPSS Statistics V 25, Armonk, NY). Descriptive results were presented as mean values and standard deviation and were depicted in graphs and tables. In the quantitative analysis, differences between the two groups were analyzed for each item using a non-parametric Mann–Whitney test. The level of significance was set at p < .05.

Results

Usability test

Twenty-three students took part in the usability test and the response rate was 95%–100%, depending on the item. Results of the usability test are presented in Table 3. For item 1 (use), item 2 (functionality) and item 3 (user-friendliness), participants awarded the app mean scores of 2.04 ± 1.02, 2.23 ± 0.81, and 2.17 ± 0.72, respectively (where 1 = very good). Students did not agree with the statement that the app was awkward (item 3) (mean score: 4.0 ± 0.82) or with the statement that the app was designed inconsistently (item 5) (mean score: 4.0 ± 1.04) (Table 3). Regarding their confidence using the app (item 6), the students gave a mean score of 3.0 ± 0.9. They were indecisive with regard to the statement that the app evaluated the students more fairly (item 7) and gave a mean score of 3.3 ± 1.15. Students agreed with the statement that the app reduces organizational effort during the clinical course (item 8) (mean score: 2.43 ± 1.12). Participants rated the app operation (item 9), design (item 10), clarity (item 11), and structure (item 12) as good to very good. The app was given an overall mean score of 2.0 ± 0.6 (item 13). Most students (81%, n = 18) said they would recommend the app after the four-week usability-test (Table 4)

Table 4.

Results of the usability test. Results are presented as relative frequencies (%) as well as the number (n), the mean values (M), and the standard deviation (SD).

Item nr.	n	M	SD
1	23	2.04	1.02
2	22	2.23	0.81
3	22	4.0	0.82
4	23	2.17	0.72
5	23	4.0	1.04
6	23	3.0	0.9
7	23	3.3	1.15
8	23	2.43	1.12
9	23	2.17	0.58
10	23	1.61	0.72
11	23	2.13	0.87
12	23	2.09	0.67
13	23	2.0	0.6
		Yes (%)	No (%)
14	22	81	19

Students’ evaluation

Table 5 shows the survey results on organization of the clinical course and documentation of the students’ performance. Seventy-five forth-year students of the integrated clinical course, who used the app for 8 weeks during their studies, and 40 fifth-year students of the same clinical course, who did not use the app, completed the evaluation. The response rate was 95% for forth-year students and 74% for fifth-year students (Figure 3). Students who used the app gave more positive assessments (higher scores) of the organization of patient treatment sessions (item 1) than students who did not use the app (p = .002). Likewise, students who used the app gave higher scores for managing and documenting treatment appointments (item 6) than students who did not use the app (p = .001). With regard to waiting time to present a treatment step to the supervisor (item 10), students who used the app gave significantly better scores than students who did not use the app (p = .038). Students who used the app also gave higher scores for treatment overview (item 15) (p = .040) and transparent documentation and assessment of course performance (item 17) (p = .012).

Table 5.

Evaluation of course organization after 8 weeks of using (IC 1) or not using (IC 2) the app. N = number, Min = minimum, Max = maximum, M = Mean, SD = standard deviation. p values were calculated using the Mann–Whitney test, IC = integrated course.

Item Nr	Course	N	Min	Max	M	Sd	p Value
1	IC 1	75	1.00	5.00	2.0133	.97943	.002*
1	IC 2	40	1.00	4.00	1.4750	.71567	.002*
2	IC 1	75	2.00	5.00	3.4400	.97593	.359
2	IC 2	39	1.00	5.00	3.5897	1.11728	.359
3	IC 1	75	1.00	5.00	1.9733	.98603	.062
3	IC 2	40	1.00	5.00	2.3250	.99711	.062
4	IC 1	75	1.00	5.00	2.3067	1.07770	.138
4	IC 2	40	1.00	5.00	2.7000	1.22370	.138
5	IC 1	74	1.00	5.00	2.0946	1.14878	.810
5	IC 2	40	1.00	5.00	2.2250	1.36790	.810
6	IC 1	75	1.00	5.00	2.6267	1.07519	.675
6	IC 2	40	1.00	5.00	2.5500	1.10824	.675
7	IC 1	75	1.00	5.00	2.2933	.99693	.001*
7	IC 2	40	1.00	5.00	3.1000	1.29694	.001*
8	IC 1	74	1.00	5.00	2.2162	.99685	.766
8	IC 2	40	1.00	5.00	2.2250	1.20868	.766
9	IC 1	74	1.00	5.00	1.9730	1.00646	.458
9	IC 2	40	1.00	5.00	1.9500	1.28002	.458
10	IC 1	74	1.00	5.00	2.6486	1.24342	.038*
10	IC 2	38	1.00	5.00	3.1842	1.24890	.038*
11	IC 1	75	1.00	4.00	1.5200	.75980	.119
11	IC 2	37	1.00	5.00	1.9459	1.22352	.119
12	IC 1	75	1.00	4.00	1.6000	.78843	.130
12	IC 2	38	1.00	4.00	1.8947	.98061	.130
13	IC 1	75	1.00	4.00	1.5733	.79140	.146
13	IC 2	38	1.00	5.00	1.8158	.92577	.146
14	IC 1	74	1.00	5.00	2.0135	1.14069	.960
14	IC 2	40	1.00	5.00	1.9250	.91672	.960
15	IC 1	75	1.00	5.00	1.8533	1.11129	.040*
15	IC 2	40	1.00	5.00	2.3250	1.26871	.040*
16	IC 1	74	1.00	5.00	2.0541	1.22633	.552
16	IC 2	40	1.00	5.00	2.1000	1.05733	.552
17	IC 1	74	1.00	5.00	1.7973	1.03341	.012*
17	IC 2	40	1.00	5.00	2.4000	1.29694	.012*

Figure 3.

Graphical representation of clinical course organization evaluation by forth-year students who used the app for 8 weeks (blue line) or did not use the app (fifth-year students, red line).

Table 6 shows the rating of the evaluation after 12 months of standard implementation conducted in the winter semester 2021/2022 and in summer semester 2022. Forty-six fifth-year students who used the app in winter semester 2021/2022 and 38 fifth-year students, who used the app in summer semester 2022, completed the evaluation. Therefore, the response rate was 61% in winter semester and 50% in summer semester. Students agreed with the statement that using the app to organize the integrated clinical course is helpful (Item 1) (Table 3) an gave a mean score of 1.57 ± 0.8 (winter semester 2021/2022) and 1.58 ± 0.79 (summer semester 2022), respectively (Table 6).

Table 6.

Results of IC 2 Evaluation (2 Items) in winter semester 2021/2022 (IC 2_WS) and in summer semester 2022 (IC 2_SoSe). N = number, Min = minimum, Max = maximum, M = Mean, SD = standard deviation. p values were calculated using the Mann–Whitney test.

Item Nr	Course	N	Min	Max	M	Sd	p Value
1	IC 2_WS	46	1,00	5,00	1,57	,807	.959
1	IC 2_SoSe	38	1,00	4,00	1,58	,793	.959
2	IC 2_WS	46	1,00	3,00	1,26	,535	.721
2	IC 2_SoSe	38	1,00	4,00	1,37	,751	.721

Regarding the statement to permanently establish the app for daily assessment and insight into course performance (item 2) (Table 3) students in the clinical course gave a mean score 1.26 ± 0.53 (winter semester 2021/2022) and 1.37 ± 0.75 (summer semester) (Table 6).

After 12 months, the fifth-year students still gave very good ratings for the organization of the treatment and the transparency of the grading in both evaluations, so that no significant changes were detectable (organization p = .959 and the rating p = .751).

App usage statistics

During 1 year (winter semester 2021/2022 and summer semester 2022) the following usage statistics were collected using the app: a total of 7605 appointments were made and 6467 treatments were started in the two integrated clinical courses of conservative and prosthetic dentistry. Students performed 85,217 treatment steps. Clinical course assistants created 6053 assessments and a total of 33,509 individual assessments.

Discussion

To the best of our knowledge, this is the first project to develop, implement, and evaluate an app for mobile devices that can help dental students to improve the organization of their clinical course and the assessment and documentation of their performance. While mobile teaching apps already exist in other areas of medicine and dentistry, this approach is unique and innovative. Our results can most be best discussed in context of the well-known experience of large companies. Entrepreneurs have recognized the potential of digitization in optimizing business processes and underlying organizational processes. Although digitization requires more resources, they found this investment is worthwhile because optimized work processes and improved organization increase productivity (Kessler et al., 2019). Increased productivity or efficiency is not easy to determine in the context of teaching because these depend on multiple factors. In this study, we evaluated the app based on a subjective evaluation of clinical course organization. One possible quantifiable parameter would be the reduction of stress after using the app. However, as causes of stress are also multifactorial, this would necessitate a cross over study design with larger cohorts and multiple surveys to exclude the many possible confounders (other sources of stress). This seemed not feasible to us against the background of the corona pandemic as well as for ethical reasons (possible (dis-) advantages of the use of the app in a critical timeframe during a mandatory course).

As preliminary research has revealed increased stress among students during the transition from pre-clinical to clinical studies (Frese et al., 2018), we selected forth-year students to test the new app. The more experienced fifth-year students continued their studies without using the app. The organizational effort and necessary documentation are similar in the forth and fifth year, therefore the results from these two groups were comparable. The first usability test showed high approval of the app (Table 3) and the majority of participants (81%) wanted to continue using the app during their clinical course.

Our results demonstrated that using the app for 8 weeks in the clinical course significantly improved the organization and documentation among forth-year students (Table 4). A survey of students in their fourth and sixth year of study at the University of Montpellier showed that perceived stress is mainly due to the many steps involved in clinical treatment and waiting for the teacher’s assessment of each step, as well as administrative tasks and computer problems (Inquimbert et al., 2017). These factors could be improved by using the new app, particularly when it comes to waiting time, which was significantly improved by the app in our study (p = .038). The waiting time to present a treatment step to the supervisor was also rated as more favourable in the first integrated course (forth year students) than in the second integrated course (fifth year students). It is conceivable that waiting times are longer in the second integrated course because of the greater complexity of the work, but at the same time this could be compensated by the greater experience of the students. Precise data on absolute waiting times have not been collected. However, this will be possible in the future through the use of the app. We expect that the objectification and transparency created by the introduction of the digital queue will increase the acceptance of waiting times and that this aspect will be perceived more positively.

The high degree of subjective stress among dentistry students is mostly caused by exam situations and practical sessions during the pre-clinical studies as well as by tests from the teachers and treatment of the patients (Alzahem et al., 2011; Elani et al., 2014). A study of 201 dentistry students in Madrid showed that experiencing high levels of stress during exam situations can be linked to lower average grades. Students who considered themselves more efficient during exam situations achieved better grades (Crego et al., 2016). The constant overview of the course program and assessments provided by the app may help dentistry students to cope with stress and the pressure to perform, which may improve their academic performance. This is supported by our finding that the increased transparency provided by the app improved students’ insight into their own performance. This was already done intensively in the past (daily documentation by course assistants, weekly course meetings, grade conferences), but remained non-transparent. Previous attempts to create this transparency, for example through standardised feedback discussions, involved a great deal of organisational effort and provided only selective information.

Our results cannot confirm a causal link between app use and improved course performance because we cannot make direct comparisons between students at different stages of their studies, whose clinical experiences and course management are different, particularly in light of the recent pandemic-related influences.

Treatment planning (FR 1), digital queues (FR 4), course evaluations (FR5), and wiki availability (FR3) are important to novices, especially when they start to treat patients. Particularly in the first clinical semester (forth-year), presenting the different treatment steps may add didactic value for students (Mayer, 2005) and help them to continue with clinical treatment processes. Before the app was introduced, a new interdisciplinary calibration of the treatment processes and requirements was initiated through focus group meetings. Lack of standardization can be stressful for students and can impair self-efficacy. A mobile learning app allows students to interact with people and course content through their mobile device, supporting formal and informal learning (Mayrberger and Bettinger, 2014; Sweeney, 2006).

The main didactic gain of this new app is the structuring, standardization, and organization of learning, treatment and assessment. A positive side-effect of the app is, that it provides a means of quality management as it represents an element of standardization that calls for continuous consensus between different dental disciplines regarding treatment procedures and assessment in clinical courses.

The following advantages arise from the perspective of the lectures: the possibility to coordinate a large number of student tandems and/or small groups and to supervise them according to their needs. The lecturer does not have to be present in the same room, but only logged into the same WLAN network. Taking this further, it is conceivable that it could be used in the practical year of the human medicine or in specialist curricula. Another advantage of the new mobile application is the elimination of the intermediate step of transferring the course results (points or grades earned) from the paper to the electronic grading system. This not only removes a potential source of error in the digitisation of course results, but also saves time in data entry and double-checking by course assistants or teaching secretaries/administration, and of course saves paper. Furthermore, the app represents a quality assurance tool, as it enables the storage of standard treatment steps as well as instrument and material lists for the integrated clinical course. The easy handling via the Web site import allows a dynamic adaptation of the teaching content via pre-formatted Excel lists. This does not require particularly high technical skills. The structure and functions of the application go beyond the clinical education of dentists and can also be adapted to pre-clinical courses and practical exercises of all kinds in many subjects. The core functions of the app correspond to the general needs in education, which are: step-by-step presentation of requirements for students, organizational efficiency, optimized communication and transparent assessment. For the pre-clinical courses (anatomy, physiology) at our Faculty of Medicine, the app is currently being adapted to facilitate course and time management, peer feedback and collaborative learning. This will hopefully contribute to a more integrated and interactive learning environment.

The mobile application was implemented at our faculty in both integrated clinical courses (in the fourth and fifth year) as standard in the winter semester 2021/2022. The evaluation after 12 months was carried out by fifth year students in the winter and summer semester confirming the organizational benefit and high value of the new application. The daily insight in study performance led to a high level of acceptance among the students with the desire to implement the mobile application permanently in the curriculum. They appreciate the benefits associated with it, such as the flexibility of scheduling appointments (booking appointments in the treatment unit from home). Our results are also in line with other studies where respondents (students, residents) like to use mobile devices and apps for time management and organisation (Paul et al., 2014; Wallace et al., 2012). The ability to have a constant overview of all their treatment appointments, as well as their course performance, is very much appreciated by the students.

Conclusion

In this project, a learning app was developed for mobile devices that optimizes organization and assessment for clinical teaching in dentistry. The app significantly improved learning processes such as organization of patient treatment with the co-workers, course documentation, waiting-time for the teacher’s assessment, the overview of treatment sessions, and assessment and grading of the students’ performance. Even after 1 year of service, the students still perceive the consistently high advantages of digitization. These findings suggest that broader use of this app should be considered to support dentistry students in their clinical and preclinical studies. After successful development and implementation of the app in the integrated clinical courses of dentistry, an application for mobile devices will now be established in cooperation with the IT Teaching (Medicine) to improve the organization and documentation for all pre-clinical internships of dental and medical students. In the new application, specifics of the dental, natural science and human medicine courses will be bundled and the interdisciplinary assessment of students as well as the performance overview and export to the campus management system will be aimed for. Future studies should focus on further development of the app to further exploit its potential.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Simona Schick

Author Biographies

Dr. med. dent. Simona Schick, MME is a research associate at the Department of Conservative Dentistry, Section of Preventive and Restorative Dentistry at the Heidelberg University Hospital. She completed her doctorate at the Heidelberg Medical Faculty in 2013, where she 2023 also successfully completed her Master of Medical Education. Since 2013, she has been working as a curriculum coordinator for the Heidelberg Medical Faculty, particularly in the field of dentistry. In this role, she is responsible for the design and implementation of the dental curriculum (HeiCuDent) in Heidelberg. Her areas of expertise include dental education, digital education, and digital dentistry. She is committed to advancing dental education through innovative methods and technologies and to ensuring that students receive a comprehensive and state-of-the-art education in this field.

Dr. med. dent. Lydia Eberhard, Research Associate and Curriculum Coordinator, Department of Prosthetic Dentistry, Heidelberg University Hospital/Heidelberg Medical Faculty.

Luis Wettach, Software Engineer, Division of Information Technology for Medical Education, Heidelberg Medical Faculty/ Heidelberg University.

Dr. med. dent. Shila Fazeli, Research Associate, Department of Conservative Dentistry, Heidelberg University Hospital.

Prof. Dr.-Ing. Andreas Heberle, Professor of Architecture and Integration of Application Systems, Faculty of Computer Science and Business Information Systems, University of Applied Sciences Karlsruhe.

Prof. Dr.- Ing. Rainer Neumann, Vice-Dean and Dean of Studies, Faculty of Computer Science and Business Information Systems, University of Applied Sciences Karlsruhe.

Jörg Miebach, Technical Director, Division of Information Technology for Medical Education, Heidelberg Medical Faculty/ Heidelberg University.

Dr. Stefan Titz, Scientific Director, Division of Information Technology for Medical Education, and curriculum coordinator preclinical Medicine (Physiology) Heidelberg Medical Faculty/ Heidelberg University.

Dr. med. dent. Jennifer Fuchs, Research Associate, Department of Oral, Dental and Maxillofacial Diseases, Heidelberg University Hospital/ Heidelberg Medical Faculty.

Prof. Dr. med. dent. Sinan Sen, Research Associate, Department of Orthodontics, Heidelberg University and since 2023 Head of Department of Orthodontics, University Clinic of Schleswig-Holstein, Campus Kiel.

Prof. Dr. med. dent. Christopher Lux, Dean of Studies (till 2023), Head of Department of Orthodontics, Heidelberg University Hospital/ Heidelberg Medical Faculty.

Prof. Cornelia Frese is Senior Consultant/Deputy Medical Director of the Polyclinic for Conservative Dentistry at Heidelberg University Hospital. She is a specialist in preventive and restorative dentistry of the German Society of Conservative Dentistry (DGZ) and a specialist in geriatric dentistry of the German Society of Geriatric Dentistry (DGAZ). Main areas of research: Oral health and prevention concepts in risk groups (e.g. centenarians and the very old), development, description, implementation and evaluation of new therapeutic procedures with direct composite materials, experimental tooth preservation (microbiology and biomaterial research). She is the winner of the German Miller Prize 2023 of the DGZMK and author of numerous continuing education articles on the above-mentioned scientific and clinical key points.

References

Al-Musawi

Al-Sane

Andersson

(2017) Smartphone App as an aid in the emergency management of avulsed teeth. Dental Traumatology: Official Publication of International Association for Dental Traumatology 33: 13–18.

Alamri

Watson

(2021) Learning technology models that support personalization within blended learning environments in higher education. TechTrends 65: 62–78.

Alzahem

van der Molen

Alaujan

, et al. (2011) Stress amongst dental students: a systematic review. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe 15: 8–18.

Anand

Chhajed

Hong

, et al. (2014) Introducing mobile technology in graduate professional education. E-Learning and Digital Media 11, 543–553.

Bates

(2015) Teaching in a Digital Age: Guidelines for Designing Teaching and Learning. Victoria: BCcampus.

Bendel

(2018) Digitalisierung. Berlin: Gabler Wirtschaftslexikon. https://wirtschaftslexikon.gabler.de/definition/digitalisierung-54195/version-384620.

Bullock

Dimond

Webb

, et al. (2015) How a mobile app supports the learning and practice of newly qualified doctors in the UK: an intervention study. BMC Medical Education 15: 71.

Crego

Carrillo-Diaz

Armfield

, et al. (2016) Stress and academic performance in dental students: the role of coping strategies and examination-related self-efficacy. Journal of Dental Education 80: 165–172.

de Oliveira

MLB

Verner

Kamburoglu

, et al. (2019) Effectiveness of using a mobile app to improve dental students' ability to identify endodontic complications from periapical radiographs. Journal of Dental Education 83: 1092–1099.

10.

Divaris

Barlow

Chendea

, et al. (2008) The academic environment: the students' perspective. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe 12(Suppl 1): 120–130.

11.

Elani

Allison

Kumar

, et al. (2014) A systematic review of stress in dental students. Journal of Dental Education 78: 226–242.

12.

Frese

Wolff

Saure

, et al. (2018) Psychosocial impact, perceived stress and learning effect in undergraduate dental students during transition from pre-clinical to clinical education. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe 22: e555–e563.

13.

Inquimbert

Tramini

Alsina

, et al. (2017) Perceived stress among French dental students and their opinion about education curriculum and pedagogy. Journal of International Society of Preventive and Community Dentistry 7: S92–S98.

14.

Kessler

Kruse

Dmitriyev

, et al. (2019) Bewertung der Digitalisierungspotenziale von Geschäftsprozessen Digitale Geschäftsmodelle–Band 1. Berlin: Springer, 83–103.

15.

Khatoon

Hill

Walmsley

(2019) Mobile learning in dentistry: challenges and opportunities. British Dental Journal 227: 298–304.

16.

Khehra

Cohenca

Cehreli

, et al. (2021) The international association of dental traumatology ToothSOS mobile app: a 2-year report. Dental Traumatology: Official Publication of International Association for Dental Traumatology 37: 145–150.

17.

Law

Thome

Lindeman

, et al. (2018) Student use and perceptions of mobile technology in clinical clerkships–Guidance for curriculum design. The American Journal of Surgery 215: 196–199.

18.

Leydon

Schwartz

(2020) Focus: medical education: the use of mobile devices to enhance engagement and integration with curricular content. Yale Journal of Biology & Medicine 93: 453–460.

19.

Mayer

(2005) Cognitive theory of multimedia learning. The Cambridge handbook of multimedia learning 41: 31–48.

20.

Mayer

(2014) The Cambridge Handbook of Multimedia Learning. Cambridge: Cambridge University Press.

21.

Mayrberger

Bettinger

(2014) Entgrenzung akademischen Lernens mit mobilen Endgeräten Nutzungspraktiken Studierender in ihrer persönlichen Lernumgebung. Jahrbuch Medienpädagogik. Berlin: Springer: 155–172.

22.

McGleenon

Morison

(2021) Preparing dental students for independent practice: a scoping review of methods and trends in undergraduate clinical skills teaching in the UK and Ireland. British Dental Journal 230: 39–45.

23.

Mittmann

Obst

(2013) Evaluierung der am Markt vorhandenen Tablets und Betriebssysteme. Münster: Bericht der Tablet-Group der Zweigbibliothek Medizin.

24.

Morrow

Pulido

Smith

, et al. (2014) Effective use of e‐grading in the dental simulation clinic. Journal of Dental Education 78: 829–837.

25.

Nayak

Sathiyabalan

, et al. (2018) Assessing the feasibility and effectiveness of an app in improving knowledge on oral cancer-an interventional study. Journal of Cancer Education: The Official Journal of the American Association for Cancer Education 33: 1250–1254.

26.

Patel

Burke-Gaffney

(2018) The value of mobile tablet computers (iPads) in the undergraduate medical curriculum. Advances in Medical Education and Practice 9: 567–570.

27.

Paul

Moussa

Asad

, et al. (2014). Medical education, PDAS and smartphones: welcome to the 21st Century. In proceedings of the Business and Health Administration Association Annual Conference 2014, Chicago, IL.

28.

Payne

KFB

Wharrad

Watts

(2012) Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Medical Informatics and Decision Making 12: 121–131.

29.

Pyorala

Maenpaa

Heinonen

, et al. (2019) The art of note taking with mobile devices in medical education. BMC Medical Education 19: 96.

30.

Rajabalee

Santally

Rennie

(2020) A study of the relationship between students’ engagement and their academic performances in an eLearning environment. E-learning and Digital Media 17: 1–20.

31.

Rungrojwittayakul

Kan

Shiozaki

, et al. (2020) Accuracy of 3D printed models created by two technologies of printers with different designs of model base. Journal of Prosthodontics: Official Journal of the American College of Prosthodontists 29: 124–128.

32.

Scheerman

JFM

van Meijel

van Empelen

, et al. (2020) The effect of using a mobile application (“WhiteTeeth”) on improving oral hygiene: a randomized controlled trial. International Journal of Dental Hygiene 18: 73–83.

33.

Schwaber

Sutherland

(2011) The scrum guide. Scrum Alliance 21: 19.

34.

Schwendicke

Samek

Krois

(2020) Artificial intelligence in dentistry: chances and challenges. Journal of Dental Research 99: 769–774.

35.

Shah

Chong

(2018) 3D imaging, 3D printing and 3D virtual planning in endodontics. Clinical Oral Investigations 22: 641–654.

36.

Sweeney

(2006) Millennial behaviors and demographics. Newark: New Jersey Institute of Technology, 10.

37.

Tang

Cao

Liu

, et al. (2021) Improving the quality of preclinical simulation training for dental students using a new digital real-time evaluation system. European Journal of Dental Education: Official Journal of the Association for Dental Education in Europe 25: 100–107.

38.

Tews

Brennan

Begaz

, et al. (2011) Medical student case presentation performance and perception when using mobile learning technology in the emergency department. Medical Education Online 16: 7327.

39.

Towers

Field

Stokes

, et al. (2019) A scoping review of the use and application of virtual reality in pre-clinical dental education. British Dental Journal 226: 358–366.

40.

Underwood

Birdsall

Kay

(2015) The use of a mobile app to motivate evidence-based oral hygiene behaviour. British Dental Journal 219: E2.

41.

Wallace

Clark

White

(2012) It's on my iPhone': attitudes to the use of mobile computing devices in medical education, a mixed-methods study. BMJ Open 2: e001099.

42.

Webb

Bullock

Dimond

, et al. (2016) Can a mobile app improve the quality of patient care provided by trainee doctors? Analysis of trainees' case reports. BMJ Open 6: e013075.

Development of a mobile app to optimize organization and assessment for clinical teaching in dentistry

Abstract

Keywords

Introduction

Emergence of digital learning across disciplines

Digitalization in dentistry and dental education

Mobile learning and digital devices

Challenges and organization of the integrated clinical courses of dentistry (Background Information)

Aim and scope of the paper

Materials and methods

Functional requirements

Project flow

Technical implementation

Administration site development and mobile devices

Usability test

Mobile application features and use (Students Version)

Treatment planning (calendar function)

Wiki

Digital Waiting List

Assessment

Mobile application features and use (Course Supervisor Version)

Overview of all treatments in the treatment units

Wiki

Digital waiting list

Assessment

Evaluation

Evaluation after 12 months of standard implementation

Statistical analysis

Results

Usability test

Students’ evaluation

App usage statistics

Discussion

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

Author Biographies

References