Smart wearable sensor-based model for monitoring medication adherence using sheep flock optimization algorithm-attention-based bidirectional long short-term memory (SFOA-Bi-LSTM)

Abstract

Objective

Medication adherence (MA) is crucial to patient treatment and vital for therapeutic outcomes. Due to its ability to continuously monitor a patient's MA behavior, the recent focus on sensor technology for MA monitoring is a promising development. The primary objective of this research is to implement sensor devices/smart wearables powered by advanced deep learning (DL) techniques to evaluate complex data patterns effectively and make accurate predictions. This study introduces a novel smart wearable sensors-based hand gesture recognition system to predict medication behaviors.

Methods

A device equipped with accelerometer and gyroscope sensors acquires and analyzes data from hand motions. A mobile app records the data from the smart device, subsequently storing it in a database in .csv file. The data is gathered, preprocessed, and classified to identify MA behavior utilizing the developed DL model known as the sheep flock optimization algorithm-attention-based bidirectional long short-term memory network (SFOA-Bi-LSTM). The data was initially gathered and preprocessed via the Z-score normalization method. The data samples are classified using the attention-based Bi-LSTM model after undergoing preprocessing. The SFOA method was utilized to optimize the hyperparameters of the attention-based Bi-LSTM model.

Results

The model's performance was examined using a five-fold cross-validation based on recall, accuracy, F1 score, and precision. The SFOA-Bi-LSTM model achieved 98.90% accuracy, 97.80% recall, 98.80% precision, and 98.62% F1 score, demonstrating its novelty and potential to inspire and motivate healthcare professionals to adopt this promising method for monitoring MA in healthcare applications.

Conclusion

The results indicate that the SFOA-Bi-LSTM model performs well in predicting MA. The SFOA-Bi-LSTM model offers several unique advantages, including efficient hyperparameter tuning via the SFOA, enhanced feature representation through an attention mechanism, and comprehensive temporal analysis using Bi-LSTM. It demonstrates superior performance compared to conventional models while being robust to noisy data due to effective preprocessing.

Introduction

Improving the average quality of life will result in continued growth in human lifespans. Recent surveys indicate a significant increase in the elderly population, particularly in developed nations. As expected, the global population of adults aged 60 and older is expected to grow by 250% in 2050 compared to 2013. ¹ This demographic shift underscores the importance of healthcare for the elderly. Furthermore, with the aging population, there is a projected increase in healthcare expenditures, specifically for long-term care. Assistive Health Technology is used to enhance the well-being of the elderly population. Significant efforts have been initiated to fulfill higher standards of excellence in healthcare systems. Innovative approaches will help to monitor human health, activities, and physical mobility. Medication adherence (MA) is essential in preserving good health and well-being, as evidenced by multiple studies emphasizing the importance of following medication instructions correctly. ¹

MA refers to the extent to which a person's behavior aligns with the prescribed guidelines of a healthcare professional, including taking medication, adhering to a particular diet, and implementing lifestyle modifications. ¹ Improving patients’ adherence to medication can significantly influence their health compared to discovering a new groundbreaking treatment, which is, therefore, closely associated with health outcomes. Moreover, inadequate or non-existent adherence was demonstrated to lead to a significant rise in death rates, diseases, and medical expenses. Although there are advantages to medications for many diseases, it was predicted that 50% of patients do not stick to their therapy in many clinical situations. This issue becomes more significant in chronic diseases requiring long-term treatments, making it less probable for patients to follow all instructions. ²

Based on data from the World Health Organization (WHO), ³ only 50% of persons with chronic illnesses adhere to the treatment guidelines despite receiving counseling on the significance of MA. Analyzing patient data using DL or machine learning (ML) techniques can help identify non-communicable illness patients who are not following their prescribed prescriptions. This early detection can potentially significantly improve patient outcomes. ³ Many technological interventions, including mobile applications that provide information about diseases and medications, reminders sent through text messages or emails, web-based systems, and electronic pill boxes for monitoring and educating about medication, are currently being created and utilized to tackle medication management. The major aim is to enhance MA. In 2018, Aldeer et al. ¹ surveyed MA technologies. They described many smart technologies for MA, such as ingestible biosensors, smart pill containers, and wearable sensors, which aid medication management. ⁴ Understanding the many causes of poor MA is necessary before developing strategies to enhance MA. As per the WHO, multiple elements contribute to inadequate adherence to medicine. ⁵ These factors are typically categorized into five groups: socioeconomic factors, patient-related factors, therapy-related factors, health system/health care team-related factors, and condition-related factors. ⁶

Multiple ways were developed to enhance or sustain patient MA. Patients have the option to utilize either compensation schemes or technology-based strategies. Compensation techniques are commonly employed as a proactive measure to prevent or delay the decline in functionality. Sometimes, they can also work as an improved alternative to existing routines, such as organizational habits. Furthermore, technology is being employed to improve adherence. ⁷ Diverse technologies integrated within digital health devices can improve patients’ health, potentially impacting the prevention and control of infectious diseases. Additionally, these devices are crucial for enhancing treatment adherence. Combining electronic health with clinical and pharmaceutical prescriptions is essential for increasing medication safety, health outcomes, and treatment. These technologies can substantially impact informing, teaching, motivating patients, and monitoring. ⁸

Several devices, such as smartphones, can be utilized in electronic health. In mobile health (mHealth), communications technologies include wearable devices (WDs) such as watches, wristbands, and text messages (short message service). mHealth pertains to using smartphones and wireless gadgets to improve health outcomes. It can address therapy adherence and education issues, offering a possible solution. ⁸ Utilizing WDs or sensors to assess and track health-related results and physical activity to sustain general health and preserve control of the motor, metabolic, cognitive performance, and cardiovascular during aging would undoubtedly provide benefits. ⁹ Without a doubt, the creation and assessment of sophisticated wearable sensor systems can produce algorithms that enable specific diagnoses and treatments, known as mHealth.^10,11

Wearable technologies are being effectively utilized in several therapeutic circumstances. By persistently monitoring medicine usage, they collect and transmit information in real time, enabling the distribution of notifications to caregivers, users, and/or healthcare providers. This ongoing follow-up, along with rapid feedback, treatment, and suggestions, signifies a novel model in healthcare. ¹² The enhanced utilization of technologies that track patient MA has shown the capacity to elevate patient adherence behaviors. ¹³ The extensive adoption of wearable sensors will enhance personalized medicine and remote healthcare from the angle of patient care. ¹⁴

Internet of Things technology can enhance MA and patient outcomes, underscoring that MA is vital and deserves equal consideration to other features of medication services. ¹⁵

This research primarily aims to enhance the accuracy of hand gesture detection by utilizing a wearable device equipped with motion sensors. Inertial sensors, often called motion sensors, can transform inertial forces into electrical signals that can be utilized to measure objects’ movement, including acceleration, tilt, and vibration. The accelerometer (ACC) and gyroscope (GYRO) were the primary inertial sensors employed to independently measure inertial acceleration and angular rate. ¹⁶ An ACC or GYRO would enable us to observe and identify movement effectively. Regarding the utilization of a solitary ACC, research has shown that it can estimate the intensities and classify the movements of multiple activities such as monitoring walking patterns, recognizing falls, and physical exertion measurement.^7,8 Due to their small dimensions, ACCs are frequently included in multiple gadgets, including smartphones and smartwatches. ¹⁷ GYROs are commonly used when the primary focus is measuring or maintaining orientation and angular rate. For instance, GYROs could be utilized to identify ankle sprains and monitor cases of falling. ¹⁸ In this work, an MA monitoring system was designed utilizing a smart wearable device to monitor a person's hand gestures while taking a pill. This paper introduces a novel method to improve the MA monitoring system by using an attention-based bidirectional long short-term memory network (Bi-LSTM) for classification and SFOAs for the optimization process.

Problem statement and scope of the research

This research addresses the challenge of accurately monitoring MA in real time, particularly through individualized tracking of medication intake behavior. Traditional methods of adherence monitoring, such as self-reporting and manual logging, are prone to inaccuracies and lack continuous, objective tracking. There is a need for a more efficient, scalable, and personalized solution that leverages advanced sensor technologies and DL models to provide accurate, real-time insights into MA, ensuring better patient outcomes and enhanced healthcare management. ¹⁹

Existing MA systems include limitations in real-time monitoring and individualized adherence tracking. Many rely on self-reports or manual logging and are prone to inaccuracies and non-adherence. Traditional methods cannot often provide continuous, real-time insights, making it challenging to detect missed doses promptly. Additionally, these systems fail to account for individual variations in behavior, gestures, or patterns, limiting their effectiveness in personalized adherence monitoring. ²⁰ These gaps highlight the need for advanced solutions using wearable sensors and DL to address these challenges.

In this research, the attention-based Bi-LSTM was implemented for its ability to capture bidirectional temporal dependencies in sequential data, making it highly effective for analyzing complex hand gesture patterns in MA monitoring. The attention mechanism further enhances the model's focus on the most relevant features, improving classification accuracy. The SFOA was selected for its efficiency in optimizing hyperparameters, which ensures the model achieves optimal performance without extensive manual tuning. Together, these methods provide a robust and accurate framework for real-time, sensor-based applications requiring precise and scalable solutions.

Research objectives

This research improves MA monitoring by developing an SFOA-Bi-LSTM methodology. The research objectives are given as follows:

To develop a smart wearable sensor-based model using a Bi-LSTM network for monitoring and predicting MA through hand gesture recognition.

To integrate the SFOA for hyperparameter tuning of the Bi-LSTM method to enhance prediction accuracy and optimize the methodology's performance.

To evaluate the performance of the SFOA-Bi-LSTM methodology by assessing its accuracy, precision, recall, and F1 score in predicting MA based on real-time sensor data.

To compare the proposed SFOA-Bi-LSTM model with existing MA systems, highlighting its advantages in real-time tracking, personalization, and accuracy.

To assess the practicality and feasibility of the proposed model for real-world healthcare applications, including its computational efficiency, resource demands, and real-time monitoring capabilities.

To explore the potential for further improvements in the model by investigating future directions for enhancing the model's adaptability, scalability, and usability in diverse patient populations.

Related works

This part analyzes recent research on MA. A smartwatch-based medication intake detection system was presented to track medication intake. ²¹ It was created using distributed ML models and data storage. The program gathers users’ activity data and transmits it to a distributed database system called Amazon Web Services. The smartwatch application utilized an ACC, nearfield communication, GYRO, and heart rate monitor. Four distinct supervised learning techniques, namely random forest, gradient-boosted trees, logistic regressions, and support vector machines (SVMs), were employed to categorize the activity of medication consumption. The study findings indicate that the gradient-boosted tree had the greatest F1 score of 0.983, while the random forest obtained the second-highest F1 score of 0.977.

A DL-based system for medicine recognition called ST-Med-Box was introduced in the research by Chang et al. ²² The system comprises an intelligent device for medicine recognition, an Android-based mobile app, a DL training server, and a cloud-based management platform. This system could identify substances and provide identification results systematically and effectively. The model can automatically deliver notifications containing the names of medications and specifying time frames for medication administration to mitigate the issue of errors in human decision-making. The problem of drug interactions resulting from the consumption of inappropriate medications was mitigated through the utilization of the ST-Med-Box. The results indicated that the recognition accuracy achieves a level of 96.6%.

A Smart Pill Box was developed to remind and monitor medication use. ²³ The technique was created to assist elderly individuals in independently managing their medication intake by ensuring they take the correct dosage at the appropriate time. The system operated in two distinct modes: Management mode and Normal mode. The remote user interface is integrated with the software programming RoboRemo to facilitate assistance from caretakers to the patient. A programmable pill box allows medical caregivers or customers to specify the quantity and timing of pill intake and the daily schedule for taking medicines. It consists of nine distinct sub-boxes. The pill dispenser was explicitly developed to mitigate errors in hospitals and retirement homes, where a substantial number of tablets must be administered to each patient daily. This feature constitutes its primary advantage.

The radiofrequency identification (RFID)-based MA intelligence system, described in the reference, was developed to promote patients’ independence and facilitate the proper use of various daily medication doses at the appropriate times. ²⁴ The system prioritized the patient's needs and was designed to be easy for users. It accomplished this by sending reminders to patients on when to take their medication and automatically dispensing it in a reliable and error-free manner. The system consisted of five components: an RFID reader, a scale, a microprocessor, a liquid crystal display, and an automatic rotation platform. The complete device model was generated using CAD design software known as SolidWorks 3D. If the patient has not consumed enough medication, the model will emit an alarm and display a text message reminding the patient to take the remaining dosage.

The paper introduced an Internet of Medical Things platform with a compact three-dimensional (3D)-printed pillbox, a remote server and database, a web application, and third-party application programming interfaces (APIs). ²⁵ The platform's objective was to improve drug adherence by offering additional features that extend the notions of personalized medicine. A functioning model that automatically accepts remote modifications was used to compare the MA of 14 participants with a dummy pillbox that sent text messages with the adjustments. The model yielded a score of 86.79, indicating outstanding user approval. The suggested affordable internet of medical things (IoMT) pillbox enhances adherence to medicine even when dealing with a complicated treatment plan while facilitating remote dosage modification.

The model in the study was developed to track the medication history of an old patient by utilizing a pill dispensing device named Fookkun and a Personal Health Record (E-MRB) to validate the information with the patient's pharmacist. ²⁶ The findings of the monitoring study, conducted over approximately six months, in a woman suffering from diabetes, age 71, were described in this document. The results indicated that Fookkun successfully promoted MA. The patient's caretaker and pharmacist do not require verification of the E-MRB as Fookkun promptly notified them once a patient failed to take medication. The primary constraint of this model was the presence of a single participant.

An eye drops medication Liquid Level Tracking System was developed in a study by Payne et al. ²⁷ A demonstration was given of a smart bottle cover that fits over a prescription eye drops medicine bottle. The intelligent cover could detect the application of eye drops, measure the quantity of fluid, and transmit usage data to a healthcare team to assist in taking action. The sleeve contains embedded electronics that measure the fluid level, the orientation of a dropper, the status (on or off), and the angular motion rate during the application. The efficacy of the sleeve was evaluated by conducting tests on a sample of 10 patients, all aged 65 or older. The sleeve could accurately detect and record 94% of the instances when it was used. The algorithm named online achieved a 0.93 (score), while the offline algorithm achieved a 0.86 F1 score.

A new medication reminder and tracking system called MedRem was introduced in a study by Abu et al. ²⁸ It operates on wearable wrist devices and integrates text-to-speech and speech recognition capabilities with intelligent interface design. Users engage with the system through spoken instructions and the device's accessible display. A dictionary-based training method was implemented on top of the most advanced speech recognition systems available to enhance the accuracy of recognizing user commands. The error rates for recognition of voice commands were 6.43% and 20.9% for native and non-native speakers, respectively, when using a commercially available speech recognition system. MedRem demonstrates exceptional accuracy in accurately identifying and interpreting users’ voice commands, with an error rate approaching zero.

An advanced system was developed to enhance the MA of individuals with hypertension, as described in the research by da Silva et al. ²⁹ The system was designed with a four-layer architecture to facilitate device integration for electronic messaging at the application layer. The first component incorporated electronic gadgets often found in modern households into the system. The next layer consisted of a preprocessing stage that selectively filtered the data derived from the activity of patients. The next layer was the reasoning component that made decisions about responding depending on the observed patient's actions. Ultimately, the final layer generates signals to influence all participants’ responses. The system was created using the J48, RandomTree, and RepTree algorithms to construct the decision layer and autonomously determine if the patient complied with the prescription. Real-time drugs were classified using artificial intelligence approaches.

An innovative device known as Time4MedTM was introduced for the electronic monitoring of MA. ³⁰ This device meets the primary requirements of affordability, simplicity of use, low intrusiveness, and small size. The analysis of adherence and the transmission of protected data through pharmacists, general practitioners, and nurses was provided. During 14 days, 20 volunteers used this device, which was programmed to deliver virtual medication thrice daily. The time and date of events recorded in the diary were compared to those preserved electronically. The device was used by 11 senior outpatients, with an average age of 80.2 ± 8.1 years, who were taking more than three medications daily over four weeks. Volunteers recorded 847 events. Functionality (100%), specificity (99.4%), sensitivity (94.9%), and recovery (100%) were all high. The system usability scale score had a mean of 82.6 (SD = 14.8), which indicates that it was highly acceptable.

A DL approach was developed to automatically classify users as adherent or non-adherent, as determined by their training actions in app utilization. ³¹ Initially, the data was prepared by implementing preprocessing and clustering procedures to group users into similar categories, focusing on the initial 90 days of exercise sessions. Then, regression models trained with users from the same cluster were employed to determine user training behavior during the fourth month using an ensemble approach. The affinity propagation clustering algorithm with the LSTM was employed to assess their adherence status. The results were the best, with an 85% F1 score and an 87% accuracy.

A novel adherence detection algorithm was developed for patients with type 2 diabetes (T2D) using DL approaches. ³² The algorithm was based on simulated continuous glucose monitoring (CGM) signals. A T2D-adapted variant of the Medtronic Virtual Patient model for type 1 diabetes simulated various CGM signals for T2D patients. Utilizing these signals, an exhaustive grid search was implemented to evaluate multiple classification algorithms. The standard logistic regression model obtained an accuracy of 65.2% ± 0.8%, which is superior to that of random. A convolutional neural network (CNN) obtained the highest classification performance, with an average accuracy of 77.5%.

In, a novel mHealth system was proposed to monitor MA in obstructive respiratory diseases. ³³ This system comprises a cloud processing module, a mobile application, and a Bluetooth acoustic sensor. The smartphone application derives Cepstogram features from audio samples from an audio sensor. The extracted features were subsequently uploaded to a cloud server, where Gaussian mixture model (GMM) classifiers were implemented to identify exhalation, inhalation drug usage, and ambient sound events. Inhaler actuation, exhalation, inhalation, and background noise were identified through data-driven and conventional classification and feature extraction methods. This model surpasses the classification accuracy of recent and pertinent existing approaches by a significant margin, reaching 98%.

Artificial Intelligence Sound Recognition on Asthma MA was developed in a study by Dimitris et al. ³⁴ This paper examines the use of ML techniques to assess asthma MA through sound pattern recognition and introduces the Drug Actuation Suite and Respiratory for benchmarking. It was supplied with a dataset that comprises drug actuation sounds and respiratory, as well as a collection of tools for feature extraction, classification procedures, and audio processing. Advanced and Conventional ML and deep network architectures were employed to implement the classification models. SVM-Mel-scale frequency cepstral coefficients (SVM-MFCC), CNN-Time, and Audiovisual Facial Action (ADA)-MFCC exhibit the maximum accuracy when patients’ data is already present in the dataset, corresponding to single and multisubject cases.

A wearable sensor system was designed for MA prediction. ³⁵ This paper introduces a two-step system for identifying the opening of a pill container using smart bottle technologies used for commercial purposes and the consumption of a pill using the smart necklace with a piezoelectric sensor. In conjunction with these two mechanisms, the mobile application can automatically monitor adherence and provide caregivers with information regarding patient status. The classification of chewable vitamins, medication capsules, saliva swallows, speaking, and consuming water was conducted using Bayesian networks, resulting in an average precision of 90.17% and recall of 88.9%. Out of a sample size of 20 participants, 135 occurrences were classified.

A smartwatch-based system was introduced to detect adherence to prescription medication by identifying multiple motions utilizing the built-in triaxial ACC and GYRO. ³⁶ The act of rotating the cap of a medicine container open and extracting a tablet or pill by pouring it into the palm were both detectable by this smartwatch. The results suggest that the algorithm's simplicity in data processing was the reason for the high accuracy of wrist rotation detection. The recall rate for opening a medication bottle was extremely high. The proposed system has significantly less human involvement than manual record keeping, other forms of adherence detection, and phone calls from nurses, even though it limits how subjects should remove the pill container to ensure successful recognition.

The use of artificial intelligence, specifically deep learning (DL) models, to monitor MA in tuberculosis (TB) treatment using video data collected from patients in Uganda was explored. ³⁷ A total of 497 annotated videos were used, with 160 frames of size 224 × 224 extracted per video. Four pretrained CNNs, including a 3D Residual Network (ResNet), were used for the binary classification of adherence. The overall, sensitivity ranged from 92.8% to 95.8%, and precision from 88% to 90.1%, indicating the model's effectiveness. However, specificity was relatively low, and external validation was not performed due to limited dataset availability.

The development of ML-based models to track MA between type 2 diabetic patients in Ethiopia was presented. ³⁸ Data from 403 patients were analyzed using General MA Scale questionnaire responses as input features. Synthetic minority oversampling technique (SMOTE) was used to balance the dataset, and eight ML models were trained and tested. Among them, the SVM model showed superior performance in recall and area under the curve (AUC). The selected model offers a promising tool for real-time monitoring of adherence behavior. However, further validation in diverse populations is required for broader applicability.

Research gap analysis

The proposed SFOA-Bi-LSTM model aims to improve on the limitations observed in the existing MA systems by providing a more personalized, real-time, and accurate approach to tracking medication intake. While previous models such as smartwatch-based systems and pill dispensers rely on basic motion detection or structured reminders, they often fail to adapt to individual user behavior, leading to suboptimal accuracy. The SFOA-Bi-LSTM model integrates attention mechanisms with Bi-LSTM, allowing it better to capture complex, temporal dependencies in user behavior, making it more effective in identifying and predicting MA patterns. Moreover, the SFOA can enhance the model's ability to fine-tune hyperparameters for improved accuracy and performance. Unlike traditional systems, which may provide limited flexibility and user personalization, this model's dynamic learning capability and attention-based architecture allow for more individualized MA monitoring. Table 1 highlights the strengths and limitations of existing models.

Table 1.

Comparison of reviewed current models.

Ref	Algorithms	Applications	Advantages	Disadvantages
21	Random forest, GBT, LR, and SVM	Smartwatch-based medication detection	High F1 score (GBT = 0.983) and real-time tracking.	Sensor dependence, high power usage.
22	Deep learning	Medication recognition with ST-Med-Box	Accuracy (96.6%), automated alerts, and reduced drug interaction errors.	Hardware-dependent, no real-time tracking.
23	Rule-based scheduling	Smart Pill Box for elderly	Remote UI, dose scheduling, and multiple compartments.	Not personalized and lacks adaptability.
24	RFID and embedded systems	Automated pill dispensing	Easy to use, reduces manual errors, and includes an alarm.	Hardware complexity and fixed schedule.
25	IoMT with cloud	Remote pillbox management	Personalized medicine, remote updates, and user-friendly.	Moderate user approval, and limited testing.
26	Rule-based monitoring	Fookkun pill dispenser with E-MRB	Real-time updates to family/pharmacist.	Single participant and lacks scalability.
27	Sensor + signal processing	Eye drops tracking system	Detection (94%), smart sleeve, and healthcare integration.	Small sample size and specific to eye drops.
28	Speech recognition	Wearable speech-based MedRem	Low error rate and interactive interface.	Higher error for non-native speakers.
29	J48, RandomTree, and RepTree	Medication tracking for hypertension	Real-time monitoring and four-layer smart system.	Algorithm performance details are limited.
30	Rule-based logging	Time4MedTM monitoring device	High usability (SUS = 82.6) and real-time logging.	Small volunteer group and simple logic.
31	Affinity propagation + LSTM	Adherence detection via app usage	F1 score (85%) and user clustering for personalized models.	Limited to app activity and not physical gestures.
32	CNN and logistic regression	T2D CGM signal-based adherence	CNN: accuracy (77.5%) and simulation-based validation.	Simulated data only and limited real-world validation.
33	GMM and signal processing	Inhaler usage monitoring (respiratory diseases)	Classification accuracy (98%) and audio-based.	May misclassify ambient noise and device-dependent.
34	SVM-MFCC and CNN-Time	Asthma MA via audio pattern recognition.	High classification accuracy and benchmark suite.	Requires dataset with audio patterns and complex processing.
35	Bayesian network	Wearable sensor-based MA (necklace and bottle)	Precision (90.17%) and two-sensor design.	Limited to small samples and potential for noise.
36	Simple motion detection (ACC/GYRO)	Smartwatch-based motion tracking	High accuracy and less human effort.	Gesture-dependent, restricted motion variability.
37	Deep learning with CNNs (including 3D ResNet)	Video-based classification of MA in TB treatment in Uganda	High sensitivity (92.8% to 95.8%) and precision (88% to 90.1%).	Low specificity (43.5% to 55.4%) and no external validation due to limited public datasets.
38	SVM	Classification and monitoring of MA in T2D patients	Achieved high recall and AUC; effective handling of imbalanced data using SMOTE.	Limited to one hospital dataset; lacks external validation for generalization.

GBT: gradient boosting trees; LR: logistic regression; SVM: support vector machine; UI: user interface; RFID: radiofrequency identification; IoMT: internet of medical things; LSTM: long short-term memory; CNN: convolutional neural network; T2D CGM: type 2 diabetes continuous glucose monitoring; GMM: Gaussian mixture model; SVM-MFCC: support vector machine-Mel-scale frequency Cepstral coefficients; MA: medication adherence; ACC: accelerometer; GYRO: gyroscope; 3D ResNet: three-dimensional residual network; TB: tuberculosis; AUC: area under the curve; SMOTE: synthetic minority oversampling technique; SUS: system usability scale.

Method

This work develops an MA monitoring system via a smart wearable sensor device. The study was conducted in Tabuk, Saudi Arabia, for six months. The smart device monitors and evaluates a patient's hand movements while ingesting a tablet. The smart device integrated sensors including a triaxial GYRO, ACC, and magnetometer to detect and accumulate data from movements of the hand for medication management. An iOS app was employed to collect gesture information from the device. The sensor's raw data are stored in the cloud in .csv format, allowing remote access and download via the app. This research created an attention-based Bi-LSTM model to classify hand movements, facilitating accurate identification of medication ingestion using sensor data. The SFOA method was utilized to optimize the hyperparameters of the attention-based Bi-LSTM model. Figure 1 illustrates the workflow of the developed paradigm.

Figure 1.

Pipeline of the proposed model.

Data acquirement

The MetaMotionRL (MMRL) device is ideally employed for recording and transmitting sensor data. A wearable device that enables real-time and continuous tracking of environmental sensor data and motion. Users can record raw data from sensors via Bluetooth at 800 Hz. The data from a sensor in an unprocessed format could be transferred at a frequency range of 100 Hz. The user could download the information as a .csv file on multiple systems such as computers, tablets, phones, cloud, or servers. In contrast, the user could download the information in a JSON format file using the APIs provided via the device. The specifications of the MMRL device and its board are shown in Figure 2.

Figure 2.

Specification of MetaMotionRL device.

The board is constructed with a small rectangular shape. It is equipped with 8 MB of NOR-FLASH memory and a combination of sensors with nine axes of motion sensing, including a 3-axis ACC, 3-axis magnetometer, and 3-axis GYRO. The sensor fusion process integrates the ACC, GYRO, and magnetometer measurements to generate a dependable accurate orientation vector in Euler or Quaternion angles. This algorithm intelligently integrates the sensor's raw data to enhance the output of each sensor. This comprises the implementation of the algorithms for calibration of each sensor's offset, the monitoring of the calibration status, and the integration of Kalman filters to generate orientation vectors that are both refined and distortion-free. A silicone slap band equipped with a rubber loop holds the sensor securely. The MMRL device is affixed or attached to a slap band for convenient handheld use, as shown in Figure 3.

Figure 3.

Slap band with MetaMotionRL device.

The application Metabases enables users to configure metasensors and retrieve sensor data. It is accessible for Android and iOS cell phones. Figure 4 displays screenshots of data collection and sensor selection executed using the MetaBase App. When connected to the app, the device tracked and recorded hand motions, including opening the pill bottle, tilting, getting a pill, tossing it into the mouth, taking a cup of water, and shutting the pill bottle. This is achieved by utilizing the sensors. The actions of these activities mentioned above are shown in Table 2.

Figure 4.

Sensors selection and data collection using the metaBase app.

Table 2.

Step-by-step actions of hand gestures monitored.

Step	Description	Activity
1	Use the right hand with a wearable device to unscrew the medicine bottle cap while holding the bottle with the left hand.
2	The action involves tilting the bottle using a handheld gadget and dispensing a pill using the other hand.
3	Putting a pill to the mouth using a hand with a handheld device.
4	Using the handheld device, pick up the glass of water, bring it to your mouth, and consume it by swallowing the liquid.
5	Close the pill bottle using the hand with a smart wearable sensor device.

The data collection process involved identifying participants who were prescribed medications taken in pill form from bottles. The participants were included after obtaining their consent, and their data were kept confidential. The hand gesture data were collected using a smart sensor device with a triaxial GYRO and ACC to capture movements. Participants are observed and recorded to ensure the consistency of their hand movements during medication intake, reducing variability in the dataset. Data collection accounted for both left-handed and right-handed participants to ensure inclusivity and comprehensiveness. The handedness and any variations in hand movements could impact the collected data. Still, these factors were mitigated through the robustness of preprocessing techniques and the research model, which effectively generalized the data patterns to classify MA activities.

Data preprocessing

Preprocessing is a crucial technique that converts input data into a structure more appropriate for integration into the research model. The raw data obtained from the dataset are considered unsuitable for further analysis because of the possible existence of duplicate, missing, or irrelevant feature data. Normalization is a data preprocessing method employed to standardize the features in a dataset to a uniform scale, enhancing the performance and precision of ML algorithms. The primary objective of normalization is to mitigate the potential biases and errors arising from the various scales of characteristics. This research uses the z-score normalization method to improve the accuracy.

Z-score normalization is a technique for normalizing data in situations where the range of the data is uncertain and has no clarity. Consequently, the range must be determined using the data's mean and standard deviation (SD). The common formula describes the function to be performed as follows:

$$Z={\displaystyle \frac{X-\overline{X}}{{\mathrm{SD}}_x}}$$

(1)

where Z defines the normalization results; X is a value to be normalized; $\overline{X}$ is an average value, and SD _x means standard deviation.

The SD can be determined by using equation (2) to take the square root of the variance of each numerical attribute.

$${\mathrm{SD}}_x=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^n{(x_i-\overline{x})}^2}{n-1}}}$$

(2)

where n is the number of samples, x denotes the average, and x_i is the value of x to i.

The z-score method normalizes all the features in the data before initiating the model's training process. ³⁹

The GYRO and ACC data were normalized using the Z-score normalization technique. Each feature was normalized independently to ensure consistent scaling across all dimensions. Normalization, specifically using the Z-score normalization technique, is applied to the ACC and GYRO data to ensure that each feature is standardized. Specifically, where X is the raw sensor value, $\overline{X}$ is the mean of the feature SD, $S{D}_x$ were computed across all samples, and normalization was performed using equation (1). This preprocessing step ensured that all features had a mean of 0 and an SD of 1, thus eliminating bias due to varying scales.

Normalization was executed in Python using the library called Scikit-learn. The StandardScaler class from the library was employed to calculate each feature's mean and SD, and the Z-score normalization method was applied. Normalization was carried out immediately after preprocessing steps, such as Z-score normalization, to remove noise and ensure that all features were on a comparable scale. The normalized data were then used as input for training the SFOA-Bi-LSTM model.

During preprocessing, missing data were addressed using an interpolation technique to estimate missing values based on neighboring data points, ensuring continuity in time-series data. Z-score normalization was employed to standardize the dataset, minimizing the impact of outliers. These steps ensured a clean and reliable dataset for training and testing, enhancing reproducibility by providing a well-documented and consistent data preparation pipeline.

To further improve data quality, outlier detection and handling were incorporated during preprocessing. Techniques such as interpolation ensured that missing values were estimated accurately, preserving the integrity of time-series data. This comprehensive preprocessing approach optimized the dataset for ML model training and enhanced overall model performance.

Classification

This research uses an attention mechanism based on a Bi-LSTM structure to enhance the model's accuracy. A recurrent neural network (RNN) is a kind of neural network that can retain short-term memory. It is characterized by its capacity to remember information and share parameters, allowing it to utilize the historical characteristics of data effectively. As a result, RNNs demonstrate superior performance in addressing time series challenges. Nevertheless, RNNs do have limitations. Specifically, when the time series delay is excessively lengthy, RNNs encounter the issue of vanishing and gradient explosion. The LSTM is a specific type of RNN that incorporates memory cells to retain significant information and discard irrelevant information. It also includes three gate cells, namely the input, forget, and output gates, which enhance the LSTM's ability to regulate the overall memory cells and retain information for an extended duration. Figure 5 depicts the LSTM cell architecture. The model's data consists of processed time series. At each moment t, $x_t$ represents data, cell state is represented by c, and h represents a hidden state. $c_{t-1}$ and $h_{t-1}$ refer to the LSTM value at the moment $t-1$ . The value of output at moments is a memory value $c_{t-1}$ and output value $h_{t-1}$ . The $f_t$ represents a forgetting gate in equation (3), which selectively filters and retains information from the preceding moment's memory unit, $c_{t-1}$ , in the unit.

$$f_t=\sigma (w_{xf}\cdot x_t+w_{hf}\cdot h_{t-1}+w_{cf}\cdot c_{t-1}+b_f)$$

(3)

Figure 5.

Framework of the long short-term memory (LSTM). ³⁹

The sigmoid activation (SA) function is represented by $\sigma$ , the bias peak is denoted as $b_f$ , and the matrix of weights between each gate and layer is represented by w. The input gate $i_t$ is responsible for selecting the relevant information to be stored. The values of the variables are regulated within the range of 0, 1 using the SA function, and the output $i_t$ , as described in equation (4):

$$i_t=\sigma (w_{xi}\cdot x_t+w_{hi}\cdot h_{t-1}+w_{ci}\cdot c_{t-1}+b_i)$$

(4)

The cell state change $c_t$ was identified by the outputs of the input and forget gates, which were shown in (5):

$$c_t=f_t{c}_{t-1}+i_t\tanh (w_{xc}\cdot x_t+w_{hc}\cdot h_{t-1}+b_c)$$

(5)

where the hyperbolic tangent activation function denotes tanh. The cell state transfer values are performed by the output gate $o_t$ using a formula outlined in equation (6):

$$o_t=\sigma (w_{xo}\cdot x_t+w_{ho}\cdot h_{t-1}+w_{co}\cdot c_{t-1}+b_o)$$

(6)

The LSTM demonstrates exceptional effectiveness in predicting irregular time series data. The Bi-LSTM network integrates the LSTM network in both forward and backward directions. This bidirectional construction allows the network output layer to effectively utilize state information from preceding and succeeding cells, enabling the network to make accurate predictions by fully leveraging the available data. The model's structure is depicted in Figure 6.

Figure 6.

The structure of the bidirectional long short-term memory (Bi-LSTM).

The neural network's hyperparameters are typically determined through manual expertise. Consequently, the research's hyperparameter selection is arbitrary, and choosing inappropriate hyperparameters might negatively impact the model's accuracy. Thus, this research selects an optimization approach to fine-tune the hyperparameters of Bi-LSTM.

Attention mechanism (AM)

The AM derives from how people consciously concentrate their attention on periods and areas among vast information to highlight crucial aspects while discarding unnecessary data. The LSTM units have constraints in preserving past information, especially when dealing with lengthy sequences. The AM effectively addresses this issue by diminishing the impact of irrelevant information, resulting in improved recall of all data. Figure 7 presents a representation that illustrates the mechanism of attention. Figure 8 depicts the architecture of the attention-based Bi-LSTM model. The AM processes the data in equations (7) to (9):

$$e_{ij}=\mathrm{score}(h_{i-1},s_j)$$

(7)

$${\alpha }_{ij}={\displaystyle \frac{\exp (e_{ij})}{{\sum }_{k=1}^n\exp (e_{ik})}}$$

(8)

$$c_i=\sum _{j=1}^n{\alpha }_{ij}{h}_i$$

(9)

Figure 7.

Structure of the attention mechanism.

Figure 8.

Architecture of attention-based bidirectional long short-term memory (Bi-LSTM) model.

The variables in the equation are defined as follows: Before time j, the LSTM's hidden state ith unit is denoted by $h_{i-1}$ , $s_j$ represents a hidden state at the current time j, $e_{ij}$ indicates the ith LSTM unit's attention weight, ${\alpha }_{ij}$ stands for the normalized weight and $c_i$ indicates the final context vector. The score() in equation (7) denotes an attention function, such as rectified linear unit (ReLU), tanh, or similar functions. This work selects the tanh function instead of ReLU. This decision is because the ReLU function can potentially eliminate several negative values during the process, which can lead to an increase in error in the projected results. ⁴⁰ The tanh function is commonly used to represent intricate nonlinear relationships, and its output is limited to the range of −1 to 1.

This research employs the AM for the Bi-LSTM layer, which assigns varying weights to data features in the network based on their significance. ⁴¹ The hyperparameters of attention-based Bi-LSTM are tuned by utilizing the sheep flock optimization algorithm (SFOA).

Tuning hyperparameters using SFOA

SFOA replicates the activities of shepherds and sheep in the pastures. The moving aspect of SFOA comprises three movements: guidance of the shepherd, interest of sheep in prior optimal experiences, and interest of sheep in joining other sheep. The grazing segment was reiterated regularly following multiple iterations of the movement segment. The sheep represents a solution, while pasture constitutes the issue's domains; the food measures at every location serve as the fitness function of the method, and the objective is to find optimal food sources. This study introduces the concept of SFOA, which is achieved by imitating the grazing behavior of sheep in a pasture to reach the highest possible food location, known as the optimum point. The sheep is a ruminant and four-legged animal commonly raised as livestock. In nature, when a sheep is taken to a pasture, it will survey its immediate surroundings. If it discovers a more favorable location for grazing, it will relocate there. Sheep strive to maximize their potential gains. Considering that the global maximum location may be beyond the grazing range of a sheep, the shepherd attempts to relocate the flock to the maximum location based on the behavior of other sheep. A sheep is motivated to continue grazing within its radius (first factor). In addition, the location of the sheep is influenced by three other factors: the shepherd's instructions, the sheep's inclination to move to a place where they have had positive experiences before, and the sheep's desire to be near other sheep. The SFOA is divided into two sections: grazing and movement. The first factor mentioned falls under the grazing section, while the other three pertain to the movement section. Goats can be found in groups of sheep, known as herds, and they exhibit a different type of locomotion. Goats exhibit higher activity levels and comprise 20% of the total population within the flock.

Algorithm steps of SFOA

The SFOA algorithm's flowchart is displayed in Figure 9. A particular character identifies every flowchart segment; additional information is provided using these identifiers. Block I defines the initial settings for parameters such as Population_size, limits, max_iteration, max_velocity, cost function, and dimensions. Population_size refers to the total number of goats and sheep in a pasture. The variable “max_iteration” represents the total quantity of iterations used for optimization. Bounds refer to each variable's upper and lower limits, representing the maximum and least feasible values. A cost function of mathematics that computes and yields the output values according to the input variables. Dimensions refer to a range of problem dimensions. The maximum velocity is determined using equation (10):

$${\mathrm{VE}}_{max}=0.1(\mathrm{UB}-\mathrm{LB})$$

(10)

Figure 9.

Sheep flock optimization (SFO) algorithm flowchart.

The lower bound (LB) and upper bound (UB) represent the limits for every variable, whereas Max_velocity $({\mathrm{VE}}_{max})$ denotes the maximum possible velocity for goats and sheep.

Block II: This block generates a random place for each member of the population and assigns a category to each member. Goats represent 20% of the members, with sheep forming the remaining members. Block III: Within this block, 10% of the iterations are allocated for additional exploration of the problem's scope. In each iteration, a location randomly was created for all the members of the populations, and its value was compared to prior values. This comparison is then used to determine the optimal amount.

Block IV: Every algorithm iteration uses the grazing condition to choose one of the two main processes. SFOA comprises two primary operations: grazing and moving. The grazing operation is carried out if the condition is met and vice versa, as represented in equation (11):

$$\mathrm{Mod}\left({\displaystyle \frac{\mathrm{iteration}\mathrm{number}}{\mathrm{maxiteration}/50}}\right)=0\mathrm{or}(\mathrm{iteration}\mathrm{number})\ge (\max \mathrm{iteration})-5$$

(11)

where Mod is the mathematical operator, iteration number defines the count of present iterations, and the maximum count of iterations is denoted by $max\mathrm{iteration}$ .

Block V: Grazing is a primary activity performed by an algorithm. Within this grazing part, each sheep feeds within a specific radius and then relocates to a more advantageous location. The values of every point of a pasture in the domain of the problem were determined by the measure of food or vegetation. The grazing radius refers to the area within which the sheep can forage. This radius is determined by the development of algorithms and is influenced by the rising environmental congestion. The grazing radius for goats and sheep is determined using equations (13) and (14), whereas the value of T is derived using equation (15). The value of $F_x$ is computed using equation (12) and then combined with the sheep's position. $\mathrm{Rand}$ is a stochastic variable that takes on values uniformly distributed between 0 and 1.

$$F_x=(2*R_{G(\mathrm{goat}\mathrm{or}\mathrm{sheep})})*Rand-R_{G(\mathrm{goat}\mathrm{or}\mathrm{sheep})}$$

(12)

$$R_{\mathrm{Gsheep}}=0.0001*(\mathrm{UB}-\mathrm{LB})*P$$

(13)

$$R_{\mathrm{Ggoat}}=0.1*(\mathrm{UB}-\mathrm{LB})*P$$

(14)

$$P=\left({\displaystyle \frac{\mathrm{iteration}\mathrm{number}}{\max \mathrm{iteration}}}\right)$$

(15)

The variable $F_x$ represents the quantity that is added to the present position. As a result, a new grazing position falls inside a grazing radius. $R_{\mathrm{Gsheep}}$ is a grazing radius of sheep and $R_{\mathrm{Ggoat}}$ is the goat's grazing radius. It is important to note that if a sheep is unable to find a more suitable grazing location, the opposite point should be checked by multiplying $F_x$ by −1 in the subsequent iteration.

Block VI: This assesses the updated positions. Once the cost is lower than the prior value of cost, the sheep will relocate to another location. Furthermore, once the value of cost is above the local or the flock sheep's global optimal cost, an updated position is allocated to them.

Block VII: Another fundamental activity of SFOA is the movement. Within the movement phase, the shepherd utilizes the globally optimal position to guide the flock toward the most advantageous location. However, due to the sheep's lack of understanding of the purpose of this directive, they continue to strive toward their individual best experiences and the proximity of other sheep. Regarding convergence, the transportation process for sheep and goats is distinct and consists of two separate sections for each animal. Move section for sheep; the iterations were categorized into dual parts: one is where convergence has still not occurred, and the flock was dispersed (P > 0.3), and the next is three phenomena that influence the movements of sheep. ⁴²

Equation (16) calculated the movement resulting from the shepherd's direction to reach the optimal position.

$$W_{\mathrm{sh}1,1}=(1-P)*C*\mathrm{Rand}(1,\mathrm{Dim})*(Y_{\mathrm{GBest}}-Y)$$

(16)

Equation (17) is the equation used to compute the movement resulting from a sheep's interest in its prior maximum experience.

$$W_{\mathrm{Lbest},1}=C*\mathrm{Rand}(1,\mathrm{Dim})*(Y_{\mathrm{LBest}}-Y)$$

(17)

The movement resulting from the sheep's desire to follow other sheep was determined using equation (18):

$$W_{\mathrm{other},1}=C*\mathrm{Rand}(1,\mathrm{Dim})*(Y_{\mathrm{Randomsheep}}-Y)$$

(18)

where Y is the current position of sheep, $Y_{\mathrm{GBest}}$ is the best fitness, $Y_{\mathrm{LBest}}$ do present sheep find the best fitness, $Y_{\mathrm{Randomsheep}}$ is the random position of sheep, and the dimension of the problem is denoted by Dim, C is determined by using equation (19); the array Rand (1, Dim) consists of random values between 0 and 1, with a size of 1 × Dim. Similarly, $W_{\mathrm{sh}1,1}$ represents the velocity at which the sheep gives orders. $W_{\mathrm{Lbest},1}$ indicates the speed at which the sheep moves toward its best experience. $W_{\mathrm{other},1}$ represents the speed at which the sheep approaches other sheep when P > 0.3.

$$C=3*\mathrm{Rand}$$

(19)

In the second portion, when the dispersion of the herd falls (P < 0.3), two things impact the movement of sheep:

Equation (20) determines the calculation of movement resulting from the sheep's instinct to choose the optimal place.

$$W_{\mathrm{sh}2,1}=C1*(1-P)*(Y_{\mathrm{LBest}}-Y)$$

(20)

The movement resulting from a sheep's interest in its past best experience is determined using equation (17):

$$\{\begin{array}{cc}{W}_{m,1}=W_{\mathrm{sh}1,1}+W_{\mathrm{Lbest},1}+W_{\mathrm{other},1}& P>0.3\\ W_{m,1}=W_{\mathrm{sh}2,1}+W_{\mathrm{Lbest},1}& P\le 0.3\end{array}\begin{array}{c}\\ \end{array}$$

(21)

$$Y_{(\mathrm{iteration}-1)}=Y_{(\mathrm{iteration})}+W_{m,1}$$

(22)

Y denotes the current position of the sheep. $Y_{\mathrm{GBest}}$ represents the best fitness achieved so far. $W_{\mathrm{sh}2,1}$ represents the speed at which the shepherd gives orders. This speed is applicable when P ≤ 0.3. $Y_{(\mathrm{iteration})}$ represents the location of all the members in all the iterations. The speed $(W_{m,1})$ obtained by the calculation using equation (21) is used to determine the next position of the sheep, as described in equation (22).

Move goat section: Within this section, all iterations are categorized into two segments. The first segment represents instances where the convergence process has not yet occurred and the goat herd is dispersed (P > 0.7). Two phenomena significantly influence the migration of goats in this scenario.

Equation (23) calculates the movement resulting from the shepherd's instruction to reach the optimal position.

$$W_{\mathrm{sh}1,2}=\mathrm{Rand}(1,\mathrm{Dim})*(Y_{\mathrm{GBest}}-Y)$$

(23)

Equation (24) is the equation used to compute the movement resulting from the goat's interest in its past best experience.

$$W_{\mathrm{Lbest},2}=(1-P)*2*\mathrm{Rand}(1,\mathrm{Dim})*(Y_{\mathrm{LBest}}-Y)$$

(24)

Let Y represent the current position of the goat. $Y_{\mathrm{GBest}}$ indicates the best fitness found so far, while $Y_{\mathrm{LBest}}$ represents the optimal fitness found by the present goat. $\mathrm{Rand}(1,\mathrm{Dim})$ was a 1*Dim array randomly ranging from zero to one. Similarly, $W_{\mathrm{sh}1,2}$ represents the speed at which the shepherd gives orders, while $W_{\mathrm{Lbest},2}$ indicates the inclination to move toward the goat's best experience when P > 0.7.

In the second portion, as the dispersion of the herd reduces (P ≤ 0.7), there is a phenomenon that influences the movement of goats.

Equation (25) determines the calculation of movement based on the shepherd's instruction to reach the optimal position.

$$W_{\mathrm{sh}2,2}=(1-P)*2*\mathrm{Rand}(1,\mathrm{Dim})*Y_{\mathrm{GBest}}-Y$$

(25)

$$\{\begin{array}{cc}{W}_{m,2}=W_{\mathrm{sh}1,2}+W_{\mathrm{Lbest},2}& P>0.7\\ W_{m,2}=W_{\mathrm{sh}2,2}& P\le 0.3\end{array}$$

(26)

$$Y_{(\mathrm{iteration}+1)}=Y_{(\mathrm{iteration})}+W_{m,2}$$

(27)

Here, the shepherd's order speed is denoted by $W_{\mathrm{sh}2,2}$ , when P ≤ 0.3. The speed $W_{m,2}$ obtained by the calculation described in equation (26) is used to determine the subsequent position of the sheep, as stated in equation (27).

Block VIII: Within this block, a new position was assessed, and the sheep were directed to move to a new location if the cost value was superior to the sheep's local and global best cost. The new position was then assigned to these variables.

Block IX represents a loop that continues for the same duration as the size of the population.

Block X represents a loop that continues for the maximum number of iterations specified. ⁴³

The SFOA optimizes hyperparameters of attention-based Bi-LSTM, such as the count of fully connected layers, the count of hidden layer units, the size of the batch, the learning rate, and the number of iterations of the neural network. During the iterations, the settings of Bi-LSTM are consistently modified until an optimal outcome is achieved. The model's error reaches a stable state after approximately 40 epochs; hence, the maximum count of epochs was fixed to 80. Due to the gradual nature of the changes, the maximum batch size is restricted to 512 to enhance the model's ability to comprehend the long-term characteristics of the information. Table 3 displays a search range and optimal value for each parameter.

Table 3.

Results of each parameter and search range in iterations.

Hyperparameters	Search ranges	Values
Epoch	[5,80]	63
Learning rate	[0.001,0.01]	0.0068
Bi-LSTM hidden layer 1	[1,30]	18
Batch size	[64,512]	336
Dense	[1100]	28
Bi-LSTM hidden layer 2	[1,30]	15

Bi-LSTM: bidirectional long short-term memory.

Performance metrics

The SFOA-Bi-LSTM model's results are assessed using recall, accuracy, F1 score, and precision. Performance evaluation is ascertained by analyzing the total true positives, true negatives, false positives, and false negatives. The predicted results have been compared with the current methodologies outlined in the literature review section to validate the efficacy of the developed model. The efficiency of the proposed model was computed utilizing the following result parameters.

Accuracy quantifies the ratio of accurate classifications out of the overall data.

$$\mathrm{Accuracy}={\displaystyle \frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}}$$

(28)

Recall is the proportion of adequately classified data to the total data analyzed.

$$\mathrm{Recall}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}}$$

(29)

Precision is determined by comparing the total accurate and incorrect classifications.

$$\mathrm{Precision}={\displaystyle \frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}}$$

(30)

The F1 score is the recall and precision harmonic mean and provides a balanced evaluation of a model's performance.

$$\mathrm{F}1\mathrm{score}={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(31)

True positive (TP) refers to the total instances of a specific attack class adequately classified. True negative (TN) represents the total instances correctly designated as not belonging to the class. False positive (FP) is the total number of cases inaccurately classified as belonging to a particular attack class when they do not. False negative (FN) is the number of cases incorrectly classified as not about a specific attack class.

High accuracy ensures most gestures are correctly classified, reducing errors in monitoring. High precision means the system rarely misidentifies non-adherence as adherence, minimizing false positives, while the F1 score highlights a balanced model that effectively handles both sensitivity and precision. These metrics ensure the system can be reliably used in real-world clinical settings for continuous, non-invasive monitoring, helping healthcare providers track MA more accurately, reduce hospital readmissions, and improve patient outcomes.

Results and performance analysis

Experiment setup

This segment presents the experiments conducted using the SFOA-Bi-LSTM approach. The acquired sensor information was utilized to assess the experiment. The SFOA-Bi-LSTM was created on Python 3.7.9 and Keras, employing TensorFlow as the backend engine. The computing environment comprised an i7 CPU, 16 GB RAM, and a Windows 10 operating system. Table 4 illustrates the sensor data acquired from the device.

Table 4.

Collected data for evaluation.

Feature	Value
Total samples	4225
No. of participants	25
No. of males	15
No. of females	10
Range of age	45–78
Range of weight	56–99 kg
Range of height	149–178 cm

Discussion on results

Within this section, the performance assessment of the SFOA-Bi-LSTM model is presented. The assessment is conducted by employing the data gathered from the wearable device using the five-fold cross-validation (CV) approach. The data set was divided into five subsets (folds) of equal size, with each fold serving as a testing set once. The remaining four folds were used for training the SFOA-Bi-LSTM model. This procedure was carried out five times, with each fold serving as the test set for a comprehensive examination. After that, the average performances over every fold were computed to provide a reliable indication of the performance of the approach. Additionally, the evaluation of the SFOA-Bi-LSTM model is dependent on metrics such as recall, accuracy, F1 score, and precision. The output of the performance assessment of the SFOA-Bi-LSTM approach calculated utilizing a five-fold CV is presented in Table 5.

Table 5.

Performance analysis of the sheep flock optimization algorithm-attention-based bidirectional long short-term memory (SFOA-Bi-LSTM) model.

Fold test	Accuracy	Sensitivity	Precision	F1 score
First fold	98.80	97.56	98.52	98.58
Second fold	98.92	97.60	98.70	98.60
Third fold	98.90	97.98	98.98	98.63
Fourth fold	99.01	97.86	98.81	98.61
Fifth fold	98.89	98.01	99	98.70
Average	98.90	97.80	98.80	98.62

A five-fold CV is crucial for evaluating the model's capacity to generalize and identifying any potential problems with overfitting or underfitting. Iteratively evaluating and refining the model using varied subsets of data aids in evaluating the model's performance on new data and yields more accurate estimations of evaluation measures. The performance evaluation of the SFOA-Bi-LSTM model demonstrates high effectiveness across all test folds, with an average accuracy of 98.90%, indicating robust overall performance. The maximum accuracy value was achieved in the fourth fold with 99.01%. The sensitivity average was approximately 97.80%. This indicates that the model is able to accurately identify approximately 97.80% of the instances that are true positives. A sensitivity rate of 98.01% was achieved in the fifth fold, which was the highest possible rate.

The model exhibits impressive precision at 98.80%, showing its reliability in correctly identifying positive instances. The precision ranges from 98.52% to 99%, where the maximum precision score was achieved in the fifth fold, at 99%. The F1 score, which averages 98.62%, highlights the model's balanced performance in precision and recall, ensuring consistent and reliable results. The fifth fold achieved the greatest F1 score of 98.70%. The results range from 98.58% to 98.70%, resulting in an average of 98.62%. Higher levels of recall, accuracy, precision, and F1 score values are generated by the model, which displays consistent performance across all folds. Figure 10 illustrates the SFOA-Bi-LSTM model's results.

Figure 10.

Graphical plot of sheep flock optimization algorithm-attention-based bidirectional long short-term memory (SFOA-Bi-LSTM) model's results.

Table 6 compares the SFOA-Bi-LSTM model's performance with several existing models. The SFOA-Bi-LSTM model attained a maximum accuracy of 98.90%, which is higher than the other models, including DL, online ML, offline rule-based ML, J48, Random tree, Rep tree, Time4Med, CNN, LSTM, CNN, GMM classifier, SVM, Bayesian classifier, and Smartwatch-based MA.^{22,23,29–35,44} The SFOA-Bi-LSTM model exhibits an impressive enhancement in accuracy than the alternative models.

Table 6.

Comparison of performance analysis.

Models	Accuracy (%)	Sensitivity (%)	Precision (%)	F1 score (%)
DL 22	96.60	NA	NA	NA
Online ML 23	NA	NA	90.00	93.00
Offline rule-based ML 23	NA	NA	80.00	92.00
J48 29	95.10	NA	92.07	NA
Random tree 29	91.50	NA	87.08	NA
Rep tree 29	98.20	NA	87.08	NA
Time4Med 30	NA	97.60	NA	NA
CNN 44	95.00	NA	NA	NA
LSTM 31	87.00	NA	94.00	85.00
CNN 32	77.50	NA	NA	NA
GMM classifier 33	98.00	NA	NA	NA
SVM 34	94.05	NA	NA	NA
Bayesian classifier 35	NA	NA	90.62	NA
Smartwatch-based MA 36	NA	NA	82.70	NA
LSTM	91.25	90.88	90.57	90.64
Bi-LSTM	93.59	93.25	93.31	93.18
Proposed SFOA-Bi-LSTM	98.90	97.80	98.80	98.62

DL: deep learning; ML: machine learning; CNN: convolutional neural network; LSTM: long short-term memory; GMM: Gaussian mixture model; SVM: support vector machine; MA: medication adherence; Bi-LSTM: bidirectional long short-term memory; SFOA-Bi-LSTM: sheep flock optimization algorithm-attention-based Bi-LSTM.

Figure 11 presents a comparison of the results. The performance analysis of accuracy across the various models highlights the exceptional performance of the SFOA-Bi-LSTM model, which achieves an impressive accuracy of 98.90%. This stands out significantly compared to other models, with the highest accuracy among those listed. For context, the DL model achieves 96.6% accuracy, the J48 model shows 95.10%, and the CNN model reports 95%.^22,29,44 In contrast, models such as LSTM and CNN exhibit lower accuracy levels of 87% and 77.5%, respectively.^31,32 This indicates that the SFOA-Bi-LSTM model excels in accuracy and is more precise than other contemporary models.

Figure 11.

Graphical plot of results comparison.

The sensitivity analysis demonstrates that the SFOA-Bi-LSTM model performs exceptionally well, with a sensitivity of 97.80%, confirming its robust capability to identify true positives accurately. Still, the Time4Med model has a sensitivity of 97.6%, which positions it near, albeit slightly lower than, the SFOA-Bi-LSTM. The SFOA-Bi-LSTM's high sensitivity indicates its strong ability to accurately identify actual positive cases, making it more useful in practical scenarios than other systems. ³⁰

The precision analysis highlights the outstanding performance of the SFOA-Bi-LSTM model, which achieves a precision of 98.80%. This indicates a high percentage of correctly identified positive examples among all the projected positives. The precision of this model is significantly higher than that of previous models, such as the Online ML model, with an accuracy of 90%, and the Bayesian classifier, with an accuracy of 90.62%.^23,35 The MA based on the Smartwatch also indicates a precision rate of 82.7%, which is relatively low. ³⁶ Although the J48 model achieves a precision of 92.07% and the Rep tree reaches 87.08% precision, they are still inferior to the precision of the SFOA-Bi-LSTM model. The model's high precision demonstrates its trustworthiness in reducing false positives and providing precise predictions, distinguishing it from other systems in the comparison.

The F1 score analysis highlights the outstanding performance of the SFOA-Bi-LSTM model, which achieves a remarkable F1 score of 98.62%. This score reflects a superior balance between precision and recall, surpassing most other models in the comparison. For example, the LSTM demonstrates an F1 score of 85%. ³¹ The offline rule-based ML attains an F1 score of 92, while the online ML model obtains 93. ²³ However, these scores are lower than the performance of the SFOA-Bi-LSTM. The high F1 score demonstrates the effectiveness of the SFOA-Bi-LSTM model in providing a well-balanced and dependable performance, positioning it as a top choice among the assessed models.

Finally, the SFOA-Bi-LSTM model attained the best accuracy, sensitivity, precision, and F1 score compared to existing models such as DL, online ML, offline rule-based ML, J48, Random tree, Rep tree, Time4Med, CNN, LSTM, CNN, GMM classifier, SVM, Bayesian classifier, and Smartwatch-based MA. The classification algorithm used improves the efficiency of the model.

The proposed SFOA-Bi-LSTM model outperformed existing models due to its advanced architecture and optimization strategy, achieving the highest accuracy (98.90%), sensitivity (97.80%), precision (98.80%), and F1 score (98.62%). Traditional ML models such as J48, Random Tree, and SVM were limited in handling complex temporal and sequential dependencies within the data, which the Bi-LSTM component effectively captured. Additionally, simpler models such as Bayesian classifiers and offline rule-based approaches lacked the adaptability and robustness required for real-world scenarios. While CNN and LSTM models performed reasonably well, their performance was hindered by less effective feature extraction and optimization than the attention-based Bi-LSTM architecture, which leveraged contextual relationships. Models such as Time4Med and smartwatch-based MA approaches were specifically designed for limited applications, resulting in lower scalability and precision when applied to diverse datasets. Incorporating the SFOA in the proposed model further enhanced hyperparameter tuning, ensuring optimal model performance, which existing models failed to achieve due to static or suboptimal configurations. This critical analysis underscores the effectiveness of the proposed SFOA-Bi-LSTM in addressing the limitations of prior methods.

Advantages and limitations

The developed SFOA-Bi-LSTM model includes several advantages such as efficient hyperparameter optimization, enhanced feature representation, bidirectional context understanding, improved performance, scalability and adaptability, and real-time monitoring capability. The SFOA-Bi-LSTM model was designed with computational efficiency, optimizing hyperparameters through SFOA to reduce training time while maintaining high accuracy. The Bi-LSTM architecture, integrated with the attention mechanism, processes data efficiently, minimizing resource demands without compromising performance. Its relatively low memory and power requirements make it suitable for deployment on WDs and real-time healthcare systems, ensuring practical utility in continuous MA monitoring while maintaining scalability and responsiveness in resource-constrained environments.

Behavioral factors such as forgetfulness, lack of motivation, or misunderstanding of medication instructions significantly impact adherence. The proposed research addresses these challenges by employing smart wearable sensors to detect hand gestures linked to medication intake, providing an automated and non-intrusive monitoring system. By focusing on real-time detection and leveraging attention-based Bi-LSTM for accurate predictions, the model accommodates varying user needs and behavioral patterns, offering flexibility and reliability to diverse patient populations.

The model's limitations include dependence on the quality and accuracy of wearable sensors, potential variability in user behavior, and challenges in handling diverse user populations with different gesture patterns, which may require further model fine-tuning for broader applicability. Potential biases in data acquisition, such as variability in hand gestures among participants due to differences in technique, speed, or physical characteristics, were mitigated by collecting a diverse dataset from participants of varying demographics. The model's robustness was enhanced, ensuring generalization across various gestures. Additionally, using an attention-based Bi-LSTM architecture allowed the model to focus on critical gesture patterns, minimizing the impact of individual variability on performance.

The SFOA-Bi-LSTM model can be integrated into existing healthcare systems through smart WDs connected to cloud platforms for real-time monitoring and analysis. Its implementation requires minimal additional infrastructure, leveraging smartphones and existing cloud services, making it cost-effective. However, potential barriers include initial investment in WDs, ensuring data privacy and compliance with regulations, and addressing resistance from healthcare professionals unfamiliar with advanced systems. Collaboration with stakeholders and demonstrating cost savings from improved MA can help overcome these challenges.

Scaling the SFOA-Bi-LSTM model for MA monitoring is economically feasible and cost-effective, offering potential savings through improved patient outcomes, reduced hospital readmissions, and lower healthcare costs. While development and operational costs include sensor hardware, cloud services, and app maintenance, the model's ability to reduce healthcare provider expenditures on in-person visits and emergencies could provide a strong return on investment. Challenges in scaling include data security, model generalization, clinical validation, regulatory approval, and market penetration. However, overcoming these challenges through clinical trials, compliance with regulations, and patient engagement strategies will help realize its full potential in healthcare systems.

Conclusion

The proposed research introduces a smart wearable sensor-based model powered by the SFOA-Bi-LSTM algorithm for monitoring MA, achieving superior results with a 98.90% accuracy, 97.80% recall, 98.80% precision, and 98.62% F1 score. By leveraging ACC and GYRO data, the model effectively recognizes hand gestures associated with medication intake, providing a robust, automated solution for adherence tracking. The use of Z-score normalization and the SFOA ensures reliable preprocessing and hyperparameter tuning, enhancing prediction accuracy. Despite limitations such as the focus on hand gestures without error detection mechanisms, the model lays a strong foundation for integrating advanced healthcare technologies. Future work will address these gaps, including dosage validation and expanded datasets, to further improve usability and accuracy. The research highlights the potential for widespread adoption in healthcare systems, aiming to improve patient outcomes by addressing non-adherence and empowering healthcare professionals with precise monitoring tools.

This research has practical implications for improving MA monitoring by enabling real-time, accurate tracking through wearable sensors and cloud-based systems, offering scalability and adaptability for diverse healthcare environments. However, limitations include dependency on sensor accuracy, potential variability in gesture recognition across different users, and challenges in ensuring privacy and security in cloud-based implementations. Future directions involve enhancing the model's robustness to diverse user behaviors and integrating advanced privacy-preserving techniques. We also plan to investigate lightweight model variants, which will improve efficiency on resource-constrained devices, thus promoting the scalability and adoption of the system in personalized healthcare applications. Paving the way for broader adoption in personalized healthcare monitoring. Lastly, we aim to test the model's performance in larger, more diverse populations to improve its generalizability and facilitate its deployment in a wider range of clinical settings.