PS -Merge operator in the classification of gait biomarkers: A preliminary approach to eXplainable Artificial Intelligence

Abstract

Background:

Many neurodegenerative diseases affect human gait. Gait analysis is an example of a non-invasive manner to diagnose these diseases. Nevertheless, gait analysis is difficult to do because patients with different neurodegenerative diseases may have similar human gaits. Machine learning algorithms may improve the correct identification of these pathologies. However, the problem with many classification algorithms is a lack of transparency and interpretability for the final user.

Methods:

In this study, we implemented the PS-Merge operator for the classification, employing gait biomarkers of a public dataset.

Results:

The highest classification percentage was 83.77%, which means an acceptable degree of reliability.

Conclusions:

Our results show that PS-Merge has the ability to explain how the algorithm chooses an option, i.e., the operator can be seen as a first step to obtaining an eXplainable Artificial Intelligence (XAI).

Keywords

Classification Neurodegenerative diseases XAI

1 Introduction

Neurodegenerative diseases are a global problem, that related with the impairment in the nervous system, producing human gait disorders, memory decline, psychopathology, depressive symptoms, sleep disorders, loss of quality of life, labor incapacity, among others afflictions [5 , 54]. There are neurodegenerative diseases that similarly affect gait, including Parkinson’s Disease (PD), Huntington’s Disease (HD), and Amyotrophic Lateral Sclerosis (ALS), which is why subtle aspects should be considered such as slow gait velocity, short step, stride lengths, among other [24, 40]. However, their classification is difficult by simple observation. Therefore, computational approaches arise as carried out in [7 , 52] to identify correctly these diseases being less invasive methods [23, 61]; demonstrating be useful to prescribe medical treatment and therapies. In addition, datasets publicly available about gait biomarkers of patients suffering from neurodegenerative diseases are emerged [20 , 64]. In this sense, we assume that the PS-merge operator is a promising option for classifying PD, HD, and ALS, but above all, because this algorithm has the added benefit that it can explain to the medical specialist how the disease classification was performed.

Belief merging employs propositional language for combining symbolic information from different sources. As has been reviewed previously in [29], the operator Δ considers as input the belief bases (profile) E, and a new consistent merged belief base Δ(E) is the output. A “belief base” a set of propositional formulae is employed to represent sources; even if the bases are inconsistent, through PS-Merge implementation can be transformed into a consistent belief base with the group representation [26]. In the literature, PS-Merge is an operator that allows for solving different decision problems [8], it has been adjusted [43], the operator has been used with two complex consensus-seeking examples [44], and has been adapted for the diagnosis of oral cancer [26]. Also, PS-Merge can offer an explainable solution to a medical specialist, since the authors consider that, partially, PS-Merge complies with the four principles of XAI defined by Phillips et al.: a) explanation, b) meaningful, c) explanation accuracy, and d) knowledge limits [41].

According to Gunning & Aha, the XAI is a novel approach to introduce novel techniques that help to offers a high degree of pattern recognition, at the same time, is also man-understandable [21]. However, a big issue is the lack of a standard legislative-ethics framework for AI technology implementation [2, 13], even when there are some governance frameworks [66], the first step to guaranteeing such a complex framework is to assure an XAI, the interpretability of XAI helps in impartiality in decision-making [3]. This research focuses on XAI implementation employing the PS-Merge operator for binary classification. The purpose of a binary classification problem is to make a prediction in which the value to be predicted can only have one of two possible values: 1 or 0, true or false, in this study, healthy or diseased. We have considered that many approaches to transforming multiclass to binary classification problems are applicable in the literature, nevertheless, one-against-one or one-versus-one (OVO) and one-against-all or one-versus-all (OVA) being the most commonly used strategies [17 , 57].

The study objective is to show that the PS-Merge may be employed to identify the neurodegenerative disease given a dataset of gait biomarkers of patients that suffer PD, HD, and ALS. Every record in the database is then considered as a belief base to be merged using PS-Merge. This research extends the work published in [51]: while the same database is employed, the format in this research is binary, with the purpose of comparing the PS-Merge operator with the Multilayer perceptron (MLP) algorithm. The method for binary transformation uses the mean, since it is the most popular and well-known [12 , 62]. Also, PS-Merge is described as an XAI.

The rest of the paper is structured as follows: Section 2 shows related work about neurodegenerative diseases classification. Section 3 depicts the materials and methods. Section 4 exposes experiments and results. Finally, Section 5 shows conclusions and future directions.

2 Related work

This section is structured into two: in the first subsection, we describe related works employing a public dataset named Gait in Neurodegenerative Disease Database (GaitND-DB) [20], which contains patients data facing PD, HD, and ALS, in the second subsection, some works are showed, employing the PS-Merge operator in classification tasks.

2.1 Experiments with GaitND-DB

Xia et al. performed experiments with support vector machine for the classification of ALS vs. healthy controls (Ctrl), achieving a 92.86 % accuracy [58].

Vipani et al. employed a Logical Regression Classifier (LRC), achieving an overall accuracy of 86.05% for multiclass classification, 87.79% for the Ctrl vs. pathological subjects classification, and e 85.22% for HD vs. PD [56].

Ye et al. using Adaptive Neuro-Fuzzy Inference System (ANFIS), obtaining 93.10% for ALS vs. Ctrl, 90.32% for PD vs. Ctrl, 94.44% for HD vs. Ctrl, and 90.63% for {ALS,HD,PD} vs. Ctrl [60].

Gupta et al. implemented a Decision tree, which trained with features of gait time series to identify the neurodegenerative disease HD vs. Ctrl, PD vs. Ctrl, and ALS vs. Ctrl obtaining results of 88.5%, 92.3%, and 96.2%, respectively [23].

Zhao et al. implemented a Long Short-Term Memory (LSTM) neural network. The best accuracies they achieved in ALS vs. Ctrl, PD vs. Ctrl, HD vs. Ctrl, and pathological subjects were 99.57%, 100%, 100%, and 97.88%, respectively [63].

Sreeja & Sujatha implemented Positive, Negative Peak Histogram Analysis (PNPHA), obtaining the best classification accuracies for Ctrl vs. ALS 90.95%, for Ctrl vs. PD 98.25%, and Ctrl vs. HD 91.25% [38].

Beyrami & Ghaderyan proposed a method based on non-negative least squares to classify ALS, PD, and HD, obtaining the best percentages of 100%, 99.78%, and 99.90%, respectively [6].

Nam Nguyen et al. used Multiscale Sample Entropy (MSE) in the data pre-processing, 10-sec time window, and the best results were with the classifier k-nearest neighbors. They obtained the best classification accuracies of 99.90%, 99.80%, 100%, 99.75%, 99.90%, 99.55%, and 99.68% for the Ctrl vs. PD, Ctrl vs. HD, Ctrl vs. ALS, PD vs. HD, PD vs. ALS, HD vs. ALS and {Ctrl vs. PD vs. HD vs. ALS}, respectively [39].

Ghaderyan & Beyrami employed sparse non-negative least squares (NNLS) to classify Ctrl vs. PD, Ctrl vs. ALS, and Ctrl vs. HD, obtaining accuracies of 98% ± 7.91, 97% ± 10.54, and 95% ± 10.54, respectively [18].

Yan et al. compared different algorithms of which, the best performance was achieved using a decision tree, obtaining for the Ctrl vs. ALS, Ctrl vs. HD, and Ctrl vs. PD, accuracies of 82.76%, 91.67%, and 90.32%, respectively [59].

Fraiwan & Hassanin compared different gait features and different algorithms, of them, the best performance was achieved using adaptive boosting (AdaBoost) obtaining for the Ctrl, ALS, PD, and HD classes, accuracies of 98.8%, 98.8%, 98.8%, and 99.4%, [15].

Erdaş et al. implemented a convolutional LSTM (ConvLSTM) to perform the multiclass classification obtaining an accuracy of 89.44%. Instead, binary classification was obtained accuracies of 96.33%, 97.68%, 94.69%, and 95.05% for pathological subjects vs. Ctrl, ALS vs. Ctrl, HD vs. Ctrl, and PD vs. Ctrl, respectively [14].

Mengarelli et al. conducted experiments employing time domain and time-dependent spectral features. They implemented k-nearest neighbor classifier, achieving 100% accuracy for binary classification and 94.84% for a 4-class classification [37].

In the previous works, we can notice that traditional algorithms have been implemented in binary and multiclass classification. In addition, it can be observed in Table 1 that acceptable results have been obtained, however, the algorithms do not explain how they obtained the results.

Table 1
Results in the state-of-the-art of classification using the GaitND-DB

Year Type of classification Reference Classifier Higher accuracy

2016 Binary Xia et al. SVM 92.86

2017 Binary Vipani et al. [56] LRC 87.79

Multiclass 86.05

2018 Binary Ye et al. [60] ANFIS 94.44

2018 Binary Gupta et al. [23] Rule-based classifier using Decision Tree 96.20

2018 Binary Zhao et al. [63] LSTM 100

2019 Binary Sreeja &Sujatha [38] PNPHA 100

2020 Multiclass Beyrami &Ghaderyan [6] Non-negative least squares 100

2020 Binary Nam Nguyen et al. [39] MSE and k-nearest neighbors 100

Multiclass 99.68

2020 Binary Ghaderyan &Beyrami [18] NNLS 98 ± 7.91

2020 Binary Yan et al. [59] Decision tree 91.67

2021 Multiclass Fraiwan &Hassanin [15] AdaBoost 99.4

2021 Binary Erdaşet al. [14] ConvLSTM 97.68

Multiclass 89.44

2022 Binary Mengarelli et al. [37] K-nearest neighbor 100

Multiclass 94.84

Year	Type of classification	Reference	Classifier	Higher accuracy
2016	Binary	Xia et al.	SVM	92.86
2017	Binary	Vipani et al. [56]	LRC	87.79
	Multiclass			86.05
2018	Binary	Ye et al. [60]	ANFIS	94.44
2018	Binary	Gupta et al. [23]	Rule-based classifier using Decision Tree	96.20
2018	Binary	Zhao et al. [63]	LSTM	100
2019	Binary	Sreeja &Sujatha [38]	PNPHA	100
2020	Multiclass	Beyrami &Ghaderyan [6]	Non-negative least squares	100
2020	Binary	Nam Nguyen et al. [39]	MSE and k-nearest neighbors	100
	Multiclass			99.68
2020	Binary	Ghaderyan &Beyrami [18]	NNLS	98 ± 7.91
2020	Binary	Yan et al. [59]	Decision tree	91.67
2021	Multiclass	Fraiwan &Hassanin [15]	AdaBoost	99.4
2021	Binary	Erdaşet al. [14]	ConvLSTM	97.68
	Multiclass			89.44
2022	Binary	Mengarelli et al. [37]	K-nearest neighbor	100
	Multiclass			94.84

2.2 Classifications with PS-Merge

Borja-Macías & Pozos-Parra defined and proposed PS-Merge operator as an approach that employs the Partial Satisfiability notion [35], also, it considers inconsistent bases and the frequency of each explicit item of beliefs, moreover, it was implemented computationally [8].

Chávez-Bosquez et al. extended PS-Merge for implementing a Belief Revision process and compared it again with two belief revision operators. Results showed a good performance of PS-Merge solving four examples of real-life [10].

Kareem et al. made the first implementation of a belief merging operator with real-world data in oral cancer diagnosis. Results showed that PS-Merge can be used for classification in the machine learning area through the correct classification percentage may be improved with extensions at the operator [26].

Pozos-Parra et al. proposed an open-source software called “Merginator”, which implements three belief merging operators (including PS-Merge). The results of this logic-based tool showed the viability of belief merging to make decisions in various study cases [44].

Velasco-Cétera et al. addressed the medical problem of identifying diabetic neuropathy in gait biomarkers. They used a dataset of 16 study subjects and implemented the PS-Merge operator for binary classification {sick, healthy}. The results showed a percentage of instances higher than 80% when using: belief merging, belief revision, and the notion of partial satisfaction [55].

3 Material and methods

3.1 Dataset

The dataset GaitND-DB [25] employed in this study contains data from subjects diagnosed with PD, HD, ALS, and Ctrl. The subjects’ age and gender distribution are shown in Table 2.

Table 2
Subjects’ age and gender distribution in GaitND-DB

Case Age groups Gender Subjects

18-29 30-39 40-49 50-59 60-69 70-79 F M number

Ctrl 6 3 2 2 2 1 14 2 16

PD 1 3 4 7 5 10 15

HD 1 5 7 4 1 2 14 6 20

ALS 2 2 2 5 2 3 10 13

Total 64

Case	Age groups	Gender	Subjects
Ctrl	6	3	2	2	2	1	14	2	16
PD		1	3	4	7	5	10	15
HD	1	5	7	4	1	2	14	6	20
ALS		2	2	2	5	2	3	10	13
	Total	64

For the construction of the GaitND-DB, Hausdorff et al. captured data from subjects walking 77 m long hallway as usual for 5 min. Force-sensitive sensors placed inside the subject’s shoes were used to capture gait data employing. A 300-Hz sampling rate with a 12-bit resolution per sample was used to capture data. In addition, 20 seconds of each record were excluded to minimize the start-up effects. The dataset contains the following 13 attributes [20]: Elapsed time, Left stride interval (seconds), Right stride interval (seconds), Left swing interval (seconds), Right swing interval (seconds), Left swing interval (% of stride), Right swing interval (% of stride), Left stance interval (seconds), Right stance interval (seconds), Left stance interval (% of stride), Right stance interval (% of stride), Double support interval (seconds), and Double support interval (% of stride).

The dataset creation process for the experiments was as follows: Raw data was downloaded from [20], then they were converted to.csv format. Subsequently, the real numbers were converted into binary 1 , adapting them for the PS-Merge algorithm.

3.2 OVO and OVA approach

The experimental design used was employing OVO and OVA, as the same that [51]:

OVO: The idea of this approach consists of a classifier for each pair of classes. Transforms a problem of c classes in c (c - 1)/2 binary problems <i, j>, one for each set of classes {i, j}, where i, j ∈ {1, …, c} and i < j. The binary classifier for an <i, j> case is training with class i and j while the samples of k (k ¬ = i, k ¬ = j) classes are ignored.

OVA: is the most popular approach, and it consists of taking a class and learning to discriminate that class from the rest. Transform a problem of c class into c binary problems, so that l = c (l represents binary training subsets). These binary problems are constructed using the i class records as positive examples and the rest as negative examples.

3.3 Attribute selection procedure and dataset subsets

Similarly to [51], the GaitND-DB with all the 13 attributes was employed, then, 5 attributes were selected based on the weights of discrete attributes given by the Chi-square method: Double support interval, Right stance interval, Left stride interval, Right stride interval, and Left stance interval. The method is one of the methods that offer better performance in the selection [32 , 65]. For this study, the data were converted to binary format, taking as a threshold the arithmetic mean of each column of the dataset GaitND-DB.

OVO and OVA approaches were implemented to obtain the datasets described below.

Datasets employing OVO: Ctrl vs. ALS, Ctrl vs. HD, Ctrl vs. PD, ALS vs. HD, ALS vs. PD and PD vs. HD;

Datasets employing OVA: Ctrl vs. {ALS,HD,PD}, ALS vs. {Ctrl,HD,PD}, HD vs. {Ctrl,ALS,PD}, and PD vs. {Ctrl,ALS,HD}.

After building the datasets, the random criterion 2/3-1/3 was used to construct the training and testing sets, given that this criterion is widely used in the literature [28, 53].

3.4 Algorithms

3.4.1 Multilayer perceptron

MLP is a feed-forward artificial neural network that contains multiple layers [45]. The MLP pseudocode is shown in Algorithm 1. The implementation of MLP was done by using the framework Waikato Environment for Knowledge Analysis (WEKA).

Algorithm 1 MLP algorithm [4, 19]

Input: Training set X⁽⁰⁾ = (x₀, x₁, x₂, …, x_{n
₀})

Initialize weights W and thresholds with the Xavier method

For every layer, perform a weighted linear summation $u_{i} (X^{(i)}) = \sum_{j = 0}^{n_{i}} w_{ij} x_{j}$

Apply the Sigmoid activation function $X^{(i + 1)} = f (u_{i} (X^{(i)})) = \frac{1}{1 + e^{- u_{i} (X^{(i)})}}$

Output: Activation of neurons in the output layer out Y = X^(out) = (y₀, y₁, y₂, …, y_m)

Weights (for (x₀, x₁, …, x_n)) are initialized using the Xavier method [19]: $Var [W^{i}] = \frac{2}{n_{i} + n_{i + 1}}$ , where: Var [Wⁱ] is the variance of the weights for the layer i, initialized with a normal distribution, and n_i, n_i+1 represents the total of neurons in the parent and the current layer. To the input values, with its assigned weight, the base function $u (X) = \sum_{i = 1}^{n} w_{i} x_{i}$ is applied. Then, the activation function is obtained, considering that, for the MLP, the sigmoidal and the hyperbolic tangent functions, are the most used activation functions [53]. They are calculated by the functions: $f_{sigm} (x) = \frac{1}{1 + e^{- x}}$ and $f_{\tanh} (x) = \frac{1 - e^{- x}}{1 + e^{- x}}$ ; they have as an image, continuous values within the intervals [0, 1] and [-1, 1]. The output is given by the function Y = F (X, W), where Y is the vector formed by the outputs (y₀, y₁, …, y_n) of the network, X is the input vector to network, and W is the weights set.

In this study, we implement the MLP algorithm, because it is a basic algorithm of the great variety that exists for classification, i.e., we use it only to compare it with the PS-Merge algorithm, since we consider that PS-Merge is in its stage initial of exploration with various datasets

3.4.2 PS-Merge

According to Borja-Macías & Pozos-Parra [8] and Kareem et al [44], a language $L$ of propositional logic is considered using a finite ordered set of symbols P : = {p₁, p₂, …, p_n} where the formulae are in Conjunctive Normal Form (CNF). According to Boufkhad & Dubois [9], a formula φ is in CNF iff φ is a conjunction of clauses, φ =C₁ ∧ … ∧ C_r, where each clause C_i is a disjunction of literals C_i = l₁ ∨ … ∨ l_k, where l_i = p_j or l_i = ¬ p_j.

A belief base K is a finite set of propositional formulae of $L$ that represents the beliefs from a source, in this research each belief base represents an observation line (record) of the dataset, which fulfills the first of the four principles of XAI: a) explanation, b) meaningful, c) explanation accuracy, and d) knowledge limits [41].

$W$ denotes the set of language models, its elements will be denoted by vectors of the form (w (p₁) , …, w (p_n)), and the models set of φ are denoted by mod (φ). K is consistent iff there exists a model of K.

A belief profile E = {K₁, …, K_m} is a multiset (bag) of m belief bases.

The classification method implements as usual the training and testing phases; the training (merging) phase is based on the following operator.

Definition 1. Let E = {K₁, …, K_m} be a belief profile and let PS-Merge be a function that maps a belief profile to a belief base, PS-Merge: $L^{n} \to L$ with the following property: the set of models of the resulting base, (i.e., the Partial Satisfiability Merge PS-Merge(E) of E) is:

${w \in W ∣ \sum_{i = 1}^{m} w_{ps} (K_{i}) \geq \sum_{i = 1}^{m} w ps' (K_{i}) for all w' \in W}$ (1) where w_ps is defined formally as follows:

Definition 2. (Normal Partial Satisfiability). For $K \in L$ , $w \in W$ , the Normal Partial Satisfiability of K for w, denoted as w_ps (K), is defined as follows:

if K ∈ P, then w_ps (K) = w (K);

if K = ¬ p, then w_ps (K) =1 - w_ps (p);

if K = l₁ ∨ … ∨ l_n, then w_ps (K) = max {w_ps (l₁) , …, w_ps (l_n)} and

if K = C₁ ∧ … ∧ C_m, then $w_{ps} (K) = \sum_{i = 1}^{m} \frac{w_{ps} (C_{i})}{m}$ .

Once obtained the merged base PS-Merge(E) by the training phase, the testing phase is performed. This second phase, in its turn, is divided into two steps. The first step uses the merged base to provide a diagnosis for every test case; however, given that there is a proportion of test cases with unknown or ambiguous diagnoses, the second step of testing was introduced to obtain a diagnosis and reduce the number of ambiguous diagnoses. For the second step, an extended version of the operator, PS-Merge_μ is needed, considering a set of belief constraints μ, where the result must satisfy the constraints, the extended operator is defined as follows:

$\begin{matrix} {w \in \mod (μ) ∣ \sum_{i = 1}^{m} w_{ps} (K_{i}) \\ \geq \sum_{i = 1}^{m} w_{ps}^{'} (K_{i}) for all w^{'} \in \mod (μ)} \end{matrix}$ (2)

The method considers every case (belief base) in the dataset as a source of information; thus, every case K is of form l₁ ∧ … ∧ l₁₃ → l₁₄ or l₁ ∧ … ∧ l₅ → l₆. In the first formula, the 13 attributes and the class are considered, in the second one only 5 attributes and the class are taken. It is in this second step of the Normal Partial Satisfiability, where the authors consider that the third of the four principles of XAI, is fulfilled: a) explanation, b) meaningful, c) explanation accuracy, and d) knowledge limits, although we know that this explanation should be made more end-user friendly, i.e. explained without mathematical formulas.

3.5 Validation metrics

In this preliminary approximation of XAI, we compare the PS-Merge operator with the MLP algorithm, in that sense, we only use the confusion matrix and the Classification rate or Accuracy (CR) since they are the metrics that consider the PS-Merge operator which is based on logic.

3.5.1 Confusion matrix

It is a metric within supervised learning to observe the behavior of the classification model. The test data allows us to see where there is confusion in the model to classify the classes correctly. In Table 3, the confusion matrix is shown, where the columns indicate classes previously labeled in the dataset and the rows the values that the model predicts. Based on the information shown in the matrix, the following can be indicated: TP: It is the number of actual or existing positive values correctly identified as positive by the model. TN: It is the number of existing negative values that were correctly classified as negative by the model. FP: It is the total of existing negative values that were incorrectly classified as positive by the model. FN: It is the total of actual or existing positive values that were incorrectly classified as negative by the model.

Table 3
Confusion matrix

ACTUAL

Positives Negatives

PREDICTED Positives TP FP

Negatives FN TN

3.5.2 Classification rate or accuracy

CR is the accuracy rate in detecting abnormal or normal behavior. It is normally used when the dataset is balanced to avoid a false sense of good model performance. $CR = \frac{TP + TN}{TP + TN + FP + FN}$

4 Experiments and results

4.1 Performance for MLP algorithm

4.1.1 OVO approach

The highest accuracy using MLP and the OVO approach for 5 attributes, was obtained for the case of Ctrl vs. HD as can be seen in Table 4. The confusion matrices for the MLP and the OVO approach using 5 and 13 attributes are given in Tables 5 and 6, respectively.

Table 4
Results of OVO approach using MLP algorithm

OVO Ctrl vs. ALS Ctrl vs. HD Ctrl vs. PD ALS vs. HD ALS vs. PD PD vs. HD

5 attributes 65.87 99.86 49.26 57.36 76.47 32.64

13 attributes 65.82 76.37 51.66 48.57 78.74 46.22

OVO	Ctrl vs. ALS	Ctrl vs. HD	Ctrl vs. PD	ALS vs. HD	ALS vs. PD	PD vs. HD
5 attributes	65.87	99.86	49.26	57.36	76.47	32.64
13 attributes	65.82	76.37	51.66	48.57	78.74	46.22

Table 5

Confusion matrices for 5 attributes in OVO approach using MLP algorithm

a=Ctrl vs. b=ALS			a=Ctrl vs. b=HD			a=Ctrl vs. b=PD
a	b	Classified as	a	b	Classified as	a	b	Classified as
966	226	a	1967	5	a	1341	631	a
447	333	b	0	1643	b	1000	243	b
a=ALS vs. b=HD			a=ALS vs. b=PD			a=PD vs. b=HD
a	b	Classified as	a	b	Classified as	a	b	Classified as
0	780	a	331	449	a	0	1243	a
253	1390	b	27	1216	b	701	942	b

Table 6

Confusion matrices for 13 attributes in OVO approach for MLP algorithm

a=Ctrl vs. b=ALS			a=Ctrl vs. b=HD			a=Ctrl vs. b=PD
a	b	Classified as	a	b	Classified as	a	b	Classified as
967	225	a	1762	210	a	1119	853	a
449	331	b	644	999	b	701	542	b
a=ALS vs. b=HD			a=ALS vs. b=PD			a=PD vs. b=HD
a	b	Classified as	a	b	Classified as	a	b	Classified as
14	766	a	418	362	a	817	426	a
480	1163	b	67	1176	b	1126	517	b

4.1.2 OVA approach

Table 7 shows that the highest accuracy of the MLP and the OVA approach based on 5 and 13 attributes, is obtained for the case of ALS vs. {Ctrl,HD,PD}, and HD vs. {Ctrl,ALS,PD} respectively. The confusion matrices for the MLP and OVA approach using 5 and 13 attributes are provided in Tables 8 and 9, respectively.

Table 7
Results of OVA approach using MLP algorithm

OVA Ctrl vs. {ALS,HD,PD} ALS vs. {Ctrl,HD,PD} HD vs.{Ctrl,ALS,PD} PD vs. {Ctrl,ALS,HD}

5 attributes 75.46 83.77 77.26 76.26

13 attributes 62.87 78.60 82.21 71.79

OVA	Ctrl vs. {ALS,HD,PD}	ALS vs. {Ctrl,HD,PD}	HD vs.{Ctrl,ALS,PD}	PD vs. {Ctrl,ALS,HD}
5 attributes	75.46	83.77	77.26	76.26
13 attributes	62.87	78.60	82.21	71.79

Table 8

Confusion matrices for 5 attributes in OVA approach for MLP algorithm

a=Ctrl vs. b={ALS,HD,PD}			a=ALS vs. b={Ctrl,HD,PD}
a	b	Classified as	a	b	Classified as
0	1192	a	0	780	a
0	3666	b	8	4070	b
a=HD vs. b={Ctrl,ALS,PD}			a=PD vs. b={Ctrl,ALS,HD}
a	b	Classified as	a	b	Classified as
374	1269	a	0	1243	a
13	3982	b	0	3995	b

Table 9

Confusion matrices for 13 attributes in OVA approach for MLP algorithm

a=Ctrl vs. b={ALS,HD,PD}			a=ALS vs. b={Ctrl,HD,PD}
a	b	Classified as	a	b	Classified as
730	1242	a	5	775	a
851	2815	b	431	4427	b
a=HD vs. b={Ctrl,ALS,PD}			a=PD vs. b={Ctrl,ALS,HD}
a	b	Classified as	a	b	Classified as
648	995	a	952	291	a
8	3987	b	1079	2536	b

4.2 Performance for PS-Merge operator

4.2.1 OVO approach

Table 10 shows the results of the PS-Merge approach for 5 and 13 attributes, the highest percentage was for case ALS vs. PD with 76.22%. The confusion matrices for the PS-Merge and the OVO approach using 5 and 13 attributes are shown in Tables 11 and 12, respectively.

Table 10
Results of OVO approach for PS-Merge algorithm

OVO Ctrl vs. ALS Ctrl vs. HD Ctrl vs. PD ALS vs. HD ALS vs. PD PD vs. HD

5 attributes 65.92 61.22 49.42 57.37 76.22 52.25

13 attributes 65.82 70.15 51.29 56.79 67.67 68.71

OVO	Ctrl vs. ALS	Ctrl vs. HD	Ctrl vs. PD	ALS vs. HD	ALS vs. PD	PD vs. HD
5 attributes	65.92	61.22	49.42	57.37	76.22	52.25
13 attributes	65.82	70.15	51.29	56.79	67.67	68.71

Table 11

Confusion matrices for 5 attributes in OVO approach for PS-Merge algorithm

a=Ctrl vs. b=ALS			a=Ctrl vs. b=HD			a=Ctrl vs. b=PD
a	b	Classified as	a	b	Classified as	a	b	Classified as
965	227	a	1967	5	a	1325	647	a
445	335	b	1397	246	b	979	264	b
a=ALS vs. b=HD			a=ALS vs. b=PD			a=PD vs. b=HD
a	b	Classified as	a	b	Classified as	a	b	Classified as
0	780	a	328	452	a	558	685	a
253	1390	b	29	1214	b	693	950	b

Table 12

Confusion matrices for 13 attributes in OVO approach for PS-Merge algorithm

a=Ctrl vs. b=ALS			a=Ctrl vs. b=HD			a=Ctrl vs. b=PD
a	b	Classified as	a	b	Classified as	a	b	Classified as
963	229	a	1905	67	a	1108	864	a
445	335	b	1012	631	b	702	541	b
a=ALS vs. b=HD			a=ALS vs. b=PD			a=PD vs. b=HD
a	b	Classified as	a	b	Classified as	a	b	Classified as
71	709	a	286	494	a	1113	130	a
338	1305	b	160	1083	b	773	870	b

4.2.2 OVA approach

In this approach, as we can see in Table 13 the highest percentages were for cases ALS vs. {Ctrl,HD,PD} with 5 attributes and 13 attributes. The confusion matrices for the PS-Merge and the OVA approach using 5 and 13 attributes are given in Tables 14 and 15 respectively.

Table 13
Results of OVA approach for PS-Merge algorithm

OVA Ctrl vs. {ALS,HD,PD} ALS vs. {Ctrl,HD,PD} HD vs.{Ctrl,ALS,PD} PD vs. {Ctrl,ALS,HD}

5 attributes 75.46 83.94 75.03 76.02

13 attributes 65.80 84.53 70.54 78.67

OVA	Ctrl vs. {ALS,HD,PD}	ALS vs. {Ctrl,HD,PD}	HD vs.{Ctrl,ALS,PD}	PD vs. {Ctrl,ALS,HD}
5 attributes	75.46	83.94	75.03	76.02
13 attributes	65.80	84.53	70.54	78.67

Table 14

Confusion matrices for 5 attributes in OVA approach for PS-Merge algorithm

a=Ctrl vs. b={ALS,HD,PD}			a=ALS vs. b={Ctrl,HD,PD}
a	b	Classified as	a	b	Classified as
0	1192	a	0	780	a
0	3666	b	0	4078	b
a=HD vs. b={Ctrl,ALS,PD}			a=PD vs. b={Ctrl,ALS,HD}
a	b	Classified as	a	b	Classified as
247	1396	a	8	1235	a
12	3983	b	21	3974	b

Table 15

Confusion matrices for 13 attributes in OVA approach for PS-Merge algorithm

a=Ctrl vs. b={ALS,HD,PD}			a=ALS vs. b={Ctrl,HD,PD}
a	b	Classified as	a	b	Classified as
759	1213	a	58	722	a
715	2951	b	150	4708	b
a=HD vs. b={Ctrl,ALS,PD}			a=PD vs. b={Ctrl,ALS,HD}
a	b	Classified as	a	b	Classified as
113	1530	a	634	609	a
131	3864	b	427	3188	b

4.3 Analysis of results for MLP and PS-Merge

4.3.1 MLP algorithm

In the case of the OVO approach, the highest classification percentage was obtained with the dataset Ctrl vs. HD: 99.86% for 5 attributes. Other competitive results were obtained for 13 attributes employing the dataset ALS vs. PD: 78.74%. In the OVA approach, we note significant results for datasets ALS vs. {Ctrl,HD,PD} with 5 attributes and HD vs. {Ctrl,ALS,PD} with 13 attributes, 83.77%, and 82.21%, respectively.

4.3.2 PS-Merge operator

Even when the results of PS-Merge do not reach the values obtained by other approaches, the method can explain why the algorithm learns or does not learn. For example, for dataset Ctrl vs. HD with 5 attributes, we have a lower percentage of 61.22% compared with 99.86% of MLP. However, it can be observed that there are opposed diagnoses generated by the MLP, v.gr.: for the 5 attributes (Double support interval, Right stance interval, Left stride interval, Right stride interval, and Left stance interval), the test records 3342 and 1754 only meet Left stride interval but the result for record 3342 is healthy and for record 1754 is diseased (see Table 16). PS-Merge in both test records provides diseased diagnosis as a result because after merging the attribute values given for the training dataset, the result for this combination of attribute values is one, i.e. 0 ∧ 0 ∧1 ∧ 0 ∧ 0 → 1. Human expert has a similar behavior when analyzing two cases with the same symptoms, the expert provides the same diagnosis, i.e, PS-Merge meets the XAI principle of explaining their rationale to a human user (in this case medical) [1] in a way more transparent [22].

The explanation for this case is given by the highest logical model, i.e. the domain explanation is not Double support interval, not Right stance interval, Left stride interval, not Right stride interval, and not Left stance interval.

Table 16
Example of contradictory test records for MLP

Test record Attribute₁ Attribute₂ Attribute₃ Attribute₄ Attribute₅ Result

1754 0 0 1 0 0 1

3342 0 0 1 0 0 0

Test record	Attribute₁	Attribute₂	Attribute₃	Attribute₄	Attribute₅	Result
1754	0	0	1	0	0	1
3342	0	0	1	0	0	0

4.3.3 Discussion

Most XAI approaches are based on machine learning or deep learning and add an additional post hoc method for interpretation. Two representative post hoc methods are: Local Interpretable Model Agnostic Explanations (LIME) [46] and SHApley Additive Explanations (SHAP) [34]. LIME has the disadvantage of providing explanations that can be really unstable. LIME introduces some randomness to the process of producing explanations given that it samples data points from a Gaussian distribution. If the sampling process is repeated a sufficient number of times, it produces different explanations for a single prediction instance. Concerning SHAP, it is computationally expensive as it computes Shapley values to several features in one prediction instance. This also makes SHAP slow and unworkable to calculate global explanations if there are many cases of prediction [27]. On the other hand, PS-Merge is a method that by construction provides an explanation; in the logic-based testing phase, PS-Merge creates an order in the logic models and then provides the highest logic models as prediction, so explanation is an internal process. The user can know the cases (the highest-ranking logical models) that support the prediction which fulfills the second of the four principles of XAI: a) explanation, b) meaningful, c) explanation accuracy and d) knowledge limits [41]. PS-Merge then does not need a post phase for a given explanation and given that it is a logic-based method, it can provide a declarative explanation. Moreover, PS-Merge is a deterministic method, then for a given dataset it always provides the same answer and the same explanation of this answer.

As a novel approach to supervised learning in the machine learning field, PS-Merge is still in its early stages of exploration. A limitation of PS-Merge is the computational cost since it orders all the logical models; however, PS-Merge can be applied to multiple domains for a small number of features or it can consider hundreds of features using a supercomputer. To advance our understanding of PS-Merge and enhance its usability, further studies are required, e.g.: computational complexity and resource consumption. As well as improvements, in order to make it more user-friendly, i. e., that the results show reflections with a simpler language giving answers to the following questions how and why are obtained. This can be achieved with front-end software development.

5 Conclusions and future work

Artificial intelligence has shown great progress with applications in medical diagnosis. However, the need for transparency and comprehensibility in the decision-making process remains paramount. In the literature, many of the results are not man-understandable, i.e. have ‘black box’ mechanisms, since the final user or specialist does not understand how the algorithm chooses an option, which could lead to wrong decisions that can be harmful [50].

In this research, we have obtained significant results in gait biomarker classification using the PS-Merge operator-based approach. It is important to highlight that we obtained 99.86% in the classification of the case Ctrl vs. HD using only 5 attributes, which is a competitive result concerning those reported in the state-of-the-art. Our PS-Merge operator-based approach provides not only high accuracy, but also a unique advantage: interpretability. It allows us to trace the logical paths leading to each classification decision, a feature that is often missing in traditional "black box" models. However, it is important to recognize that our approach did not consistently outperform other methods in terms of precision, which highlights the need for further refinement 2 . The requirement of binary input in PS-Merge is a great limitation because by transforming real numbers to binary ones the search space is reduced and then the solution is not as fine as the original domain. If we analyze the binary dataset, we can observe that the use of the arithmetic mean for the binarization introduces a high imprecision in the new dataset, in consequence, the small accuracy can be avoided using a better criterion of binarization. However, we believe that a mixed approach, which uses the advantages of the two main approaches of AI (connectionist and logic-based), will allow us to create a robust XAI. We must remember that logic-based approaches allowed creating expert systems that allowed tracing the premises to have an explanation of a given conclusion. We believe that PS-Merge combine with machine learning o deep learning methods will allow us to obtain a mixed approach that provides intuitive and robust explanations.

This first approximation to the XAI based on the PS-Merge operator in the classification of gait biomarkers is important, because as Samek et al. say: “the ability to explain the rationale behind one’s decisions to other people is an important aspect of human intelligence”, and in some cases, it is a prerequisite, e.g., when a medical doctor explains the therapy decision to the patients [50].

In future work, we propose the study of classification as follows:

To transform ordinal values to binary, one-hot strategy can be used.

To implement the PS-Merge operator with real datasets from other areas.

To implement other merging operators and compare the results.

Footnotes

GaitND-DB dataset available from: .

Here, we recognize that the fourth of the four principles of XAI, is fulfilled: a) explanation, b) meaningful, c) explanation accuracy and d) knowledge limits.

References

Amina Adadi , Mohammed Berrada , Peeking inside the black-box: Asurvey on Explainable Artificial Intelligence (XAI), IEEEAccess 6 (2018), 52138–52160.

Moran-Reyes Ariel Antonio , Towards an ethical framework about bigdata era: metaethical, normative ethical and hermeneuticalapproaches, Heliyon 8(2) (2022), e08926.

Alejandro Barredo Arrieta , Natalia Díaz-Rodríguez , Javier Del Ser , Adrien Bennetot , Siham Tabik , Alberto Barbado , Salvador Garcia , Sergio Gil-Lopez, , Daniel Molina , Richard Benjamins et al. Explainable artificial intelligence (xai): Concepts, toaxonmies,opportunities and challenges toward responsible ai, InformationFusion 58 (2020), 82–115.

Jason Baldridge, Simple code for trying out TensorFlow with simulated datasets. GitHub Inc., 2015. ANN algorithm available at https://github.com/jasonbaldridge/trytf/blob/master/hidden.py

Miquel Baquero , Nuria Martín , Depressive symptoms inneurodegenerative diseases, World Journal of Clinical Cases:WJCC 3(8) (2015), 682.

Seyede Marziyeh Ghoreshi Beyrami , Peyvand Ghaderyan , A robust,cost-effective and non-invasive computer-aided method for diagnosisthree types of neurodegenerative diseases with gait signal analysis, Measurement 156 (2020), 107579.

Suleyman Bilgin

, The impact of feature extraction for theclassification of amyotrophic lateral sclerosis amongneurodegenerative diseases and healthy subjects, BiomedicalSignal Processing and Control 31 (2017), 288–294.

Veróonica Borja Macías , Pilar Pozos Parra Implementing PS-Merge operator. In Mexican International Conference on Artificial Intelligence, pages 39–50. Springer, 2009.

Yacine Boufkhad , Olivier Dubois , Length of prime implicants andnumber of solutions of random cnf formulae, TheoreticalComputer Science 215(1-2) (1999), 1–30.

10.

Oscar Chávez-Bosquez , Pilar Pozos-Parra , Jianbing Ma,Implementing , Δps (PS-Merge) Belief Merging Operator forBelief Revision,n y Sistemas, Computaci’o 21(3) (2017), 419–434.

11.

Sáenz de Pipaón

and Larumbe

, Programa de enfermedadesneurodegenerativas, In Anales del Sistema Sanitario de Navarra 24 (2001), 49–76.

12.

Kai-Bo Duan , Jagath Rajapakse

, Minh N. Nguyen Oneversus- one and one-versus-all multiclass svm-rfe for gene selection in cancer classification. In European conference on evolutionary computation, machine learning and data mining in bioinformatics, pages 47–56. Springer, 2007.

13.

González-Esteban Elsa , Patrici Calvo , Ethically governingartificial intelligence in the field of scientific research andinnovation, Heliyon 8(2) (2022), e08946.

14.

Çağatay Berke Erdaş , Emre Sümer , SedaKibaroğlu Neurodegenerative disease detection and severityprediction using deep learning approaches, Biomedical SignalProcessing and Control 70 (2021), 103069.

15.

Luay Fraiwan , Omnia Hassanin , Computer-aided identification ofdegenerative neuromuscular diseases based on gait dynamics andensemble decision tree classifiers, Plos One 16(6) (2021), e0252380.

16.

Mirta Fuentes-Ramos , Eddy Sánchez-DelaCruz , Iván-VladimirMeza-Ruiz , Iván-VladimirMeza-Ruiz , Iván-VladimirMeza-Ruiz , Neurodegenerativediseases categorization by applying the automatic model selectionand hyperparameter optimization method, Journal of Intelligent& Fuzzy Systems 42(5) (2022), 4759–4767.

17.

Mikel Galar , Alberto Fernández , Edurne Barrenechea, , HumbertoBustince , Francisco Herrera , An overview of ensemble methods forbinary classifiers in multi-class problems: Experimental study onone-vs-one and one-vs-all schemes, Pattern Recognition 44(8) (2011), 1761–1776.

18.

Peyvand Ghaderyan , Seyede Marziyeh Ghoreshi Beyrami , Neurodegenerative diseases detection using distance metrics andsparse coding: A new perspective on gait symmetric features, Computers in Biology and Medicine 120 (2020), 103736.

19.

Xavier Glorot , Yoshua Bengio Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256, 2010.

20.

Goldberger

A.L.

, Amaral

L.A.N.

, Glass

, Hausdorff

J.M.

, Ch.Ivanov

, Mark

R.G.

, Mietus

J.E.

, Moody

G.B.

, Peng

C.-K.

and Stanley

H.E.

, Physiobank, Physiotoolkit, and Physionet: Components of aNew Research Resource for Complex Physiologic Signals, Circulation 101(23) (2000), e215–e220. Circulation Electronic Pages: https://circ.ahajournals.org/content/101/23/e215.full.pdf+html?sid=9dfce19a-f77d-4d0b-96d6-9c9e819ea980. Database: https://www.physionet.org/physiobank/database/gaitndd/

21.

David Gunning Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), nd Web, 2017. Available: https://nsarchive.gwu.edu/sites/default/files/documents/5794867/National-Security-Archive-David-Gunning-DARPA. pdf

22.

David Gunning and David Aha

, Darpa’s Explainable ArtificialIntelligence Program, AI Magazine 40(2) (2019), 44–58.

23.

Kartikay Gupta , Aayushi Khajuria , Niladri Chatterjee , Pradeep Joshiand Deepak Joshi , Rule based classification of neurodegenerativediseases using data driven gait features, Health andTechnology (2018), 1–14.

24.

Chris Hass,

, Paul Malczak , Joe Nocera , Elizabeth Stegemoller

, Aparna Shukala, , Quantitative normative gait data in a large cohortof ambulatory persons with Parkinson’s disease, PloS One 7(8) (2012), e42337.

25.

Jeffrey Hausdorff

, Zvi Ladin and Jeanne Wei

, Footswitch systemfor measurement of the temporal parameters of gait, Journal ofBiomechanics 28(3) (1995), 347–351.

26.

Sameem Abdul Kareem , Pilar Pozos-Parra , Nic Wilson , Anapplication of belief merging for the diagnosis of oral cancer, Applied Soft Computing 61 (2017), 1105–1112.

27.

Samanta Knapič , Avleen Malhi , Rohit Saluja , KaryFrämling , Explainable artificial intelligence for human decisionsupport system in the medical domain, Machine Learning andKnowledge Extraction 3(3) (2021), 740–770.

28.

Ron Kohavi A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI, volume 14, pages 1137–1145. Stanford, CA, 1995.

29.

Sébastien Konieczny , Ramón Pino Pérez , and , Logic basedmerging, Journal of Philosophical Logic 40(2) (2011), 239–270.

30.

Robert Levenson

, Virginia Sturm

and Claudia Haase

, Emotional and behavioral symptoms in neurodegenerative disease: amodel for studying the neural bases of psychopathology, AnnualReview of Clinical Psychology 10 (2014), 581.

31.

Qianmu Li , Yanjun Song , Jing Zhang and Victor Sheng,

, Multiclassimbalanced learning with one-versus-one decomposition and spectralclustering, Expert Systems with Applications 147 (2020), 113152.

32.

Huiqing Liu , Jinyan Li , Limsoon Wong , A comparative study onfeature selection and classification methods using gene expressionprofiles and proteomic patterns, Genome informatics 13(2002), 51–60.

33.

Liu

and Schumann

, Data mining feature selection for creditscoring models,, Journal of the Operational Research Society 56(9) (2005), 1099–1108.

34.

Scott Lundberg

, Su-In Lee , A unified approach to interpretingmodel predictions, Advances in Neural Information ProcessingSystems 30 (2017).

35.

Verónica Borja Macías , Pilar Pozos Parra Model-based belief merging without distance measures. In Sixth International Conference on Autonomous Agents and Multiagent Systems, pages 613–615. ACM, 2007.

36.

Yasushi Makihara , Hidetoshi Mannami , Akira Tsuji , Md Altab Hossain , Kazushige Sugiura , Atsushi Mori , Yasushi Yagi , The OU-ISIR gaitdatabase comprising the treadmill dataset, IPSJ Transactions onComputer Vision and Applications 4(0) (2012), 53–62. https://www.am.anken.osaka-u.ac.jp/BiometricDB/index.html

37.

Alessandro Mengarelli , Andrea Tigrini, Sandro Fioretti , FedericaVerdini , Identification of neurodegenerative diseases from gaitrhythm through time domain and time-dependent spectral descriptors, IEEE Journal of Biomedical and Health Informatics 26(12) (2022), 5974–5982.

38.

Sreeja Mole

S.S.

and Sujatha

, An efficient gait dynamicsclassification method for neurodegenerative diseases using brainsignals, Journal of Medical Systems 43(8) (2019), 1–10.

39.

Quoc Duy Nam Nguyen , An-Bang Liu , Che-Wei Lin , Development of aneurodegenerative disease gait classification algorithm usingmultiscale sample entropy and machine learning classifiers, Entropy 22(12) (2020), 1340.

40.

Alberto Isaac Pérez-Sanpablo , Arturo González-Mendoza , IvettQuiñones-Uriostegui , Gerardo Rodríguez-Reyes , LidiaNúñez-Carrera , Claudia Hernández-Arenas , Marie CatherineBoll-Woehrlen , Aldo Alessi Montero , Evidence-based design anddevelopment of a VR-based treadmill system for gait research andrehabilitation of patients with Parkinson’s disease, Revista de investigacion clinica; organo del Hospital de Enfermedades de la Nutricion 66 (2014), s39–s47.

41.

Jonathon Phillips

, Carina Hahn,

, Peter Fontana

, David Broniatowski

, Mark Przybocki

Four principles of explainable artificial intelligence, Technical report, Gaithersburg, Maryland, 2020.

42.

Ellen Pitt , Richi Nayak The use of various data mining and feature selection methods in the analysis of a population survey dataset. In Proceedings of the 2nd international workshop on Integrating artificial intelligence and data mining-Volume 84, pages 83–93. Australian Computer Society, Inc., 2007.

43.

Pilar Pozos-Parra, , Laurent Perrussel , Jean Marc Thevenin Belief merging using normal forms. In Mexican International Conference on Artificial Intelligence, pages 40–51. Springer, 2011.

44.

Pilar Pozos-Parra, , Oscar Chávez-Bosquez , Kevin McAreavey , Merginator: A belief merging tool for consensus support, Journal of Intelligent and Fuzzy Systems 34(5) (2018), 3199–3210.

45.

Peng Ren , Shanjiang Tang , Fang Fang , Lizhu Luo , Lei Xu , Maria LBringas-Vega , Dezhong Yao , Keith M Kendrick , Pedro A Valdes-Sosa , Gait rhythm fluctuation analysis for neurodegenerative diseases byempirical mode decomposition, IEEE Transactions on BiomedicalEngineering 64(1) (2017), 52–60.

46.

Marco Tulio Ribeiro , Sameer Singh , Carlos Guestrin, why should i trust you? explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016.

47.

Anderson Rocha , Siome Klein Goldenstein , Multiclass from binary:Expanding one-versus-all, one-versus-one and ecoc-based approaches, IEEE Transactions on Neural Networks and Learning Systems 25(2) (2013), 289–302.

48.

49.

Masume Saljuqi , Peyvand Ghaderyan , A novel method based onmatching pursuit decomposition of gait signals for parkinson’sdisease, amyotrophic lateral sclerosis and huntington’s diseasedetection, Neuroscience Letters 761 (2021), 136107.

50.

Wojciech Samek , Wiegand , Klaus-Robert Muller Explainable artificial intelligence: Uderstanding, visualizing and interpreting deep learning models. arXivpreprint arXiv:1708.08296, 2017.

51.

Eddy Sanchez-DelaCruz , Pilar Pozos-Parra , Machine learning-basedclassification for diagnosis of neurodegenerative diseases, In Proceedings of the Eleventh Latin American Workshop on New Methods of Reasoning 2264 (2018), 40–50. CEURS.org.

52.

Eddy Sáanchez-DelaCruz , Francisco Acosta-Escalante , CatherineBoll-Woehrlen , Francisco Álvarez-Rodríguez

, AdánHernández-Nolasco , Miguel Wister

, Pablo Pancardo , Categorización de enfermedades neurodegenerativas a partir debiomarcadores de la marcha, Komputer Sapiens 2 (2015), 16–19.

53.

Eddy Sánchez-DelaCruz , Roberto Weber , Biswal

R.R.

, Jose Mejia , Gandhi Hernandez-Chan , Heberto Gomez-Pozos , Gait biomarkersclassification by combining assembled algorithms and deep learning:results of a local study, Computational and MathematicalMethods in Medicine 2019 (2019).

54.

Alexandra-Maria Tăuţan , Bogdan Ionescu , EmilianoSantarnecchi , Artificial intelligence in neurodegenerative diseases:A review of available tools with a focus on machine learningtechniques, Artificial Intelligence in Medicine 117(2021), 102081.

55.

América Velasco-Cétera , Eddy Sánchez-DelaCruz , PilarPozos-Parra , Clasificación binaria de neuropatíadiabética implementando el operador PS-Merge, Research inComputing Science 149(8) (2020), 1133–1146.

56.

Raj Vipani , Sambit Hore , Souryadeep Basak , Saibal Dutta Gait signal classification tool utilizing hilbert transform based feature extraction and logistic regression based classification. In 2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), pages 57–61. IEEE, 2017.

57.

Yi Xia , Qingwei Gao , Qiang Ye , Classification of gait rhythmsignals between patients with neuro-degenerative diseases and normalsubjects: experiments with statistical features and differentclassification models, Biomedical Signal Processing andControl 18 (2015), 254–262.

58.

Yi Xia , Qingwei Gao , Yixiang Lu , Qiang Ye , A novel approach foranalysis of altered gait variability in amyotrophic lateralsclerosis, Medical & Biological Engineering & Computing 54(9) (2016), 1399–1408.

59.

Yan Yan , Kamen Ivanov , Olatunji Mumini Omisore , Tobore Igbe , QiuhuaLiu , Zedong Nie , Lei Wang , Gait rhythm dynamics forneuro-degenerative disease classification via persistencelandscape-based topological representation, Sensors 20(7) (2020), 2006.

60.

Qiang Ye , Yi Xia , Zhiming Yao Classification of gait patterns inpatients with neurodegenerative disease using adaptive neuro-fuzzyinference system, Computational and Mathematical Methods inMedicine 2018 (2018).

61.

Wei Zeng , CongWang , Classification of neurodegenerative diseasesusing gait dynamics via deterministic learning, InformationSciences 317 (2015), 246–258.

62.

Zhong-Liang Zhang , Xing-Gang Luo , Salvador Garcia , Jia-Fu Tang , Francisco Herrera , Exploring the effectiveness of dynamic ensembleselection in the one-versus-one scheme, Knowledge-BasedSystems 125 (2017), 53–63.

63.

Aite Zhao , Lin Qi , Junyu Dong , Hui Yu , Dual channel lstm basedmulti-feature extraction in gait for diagnosis of neurodegenerativediseases, Knowledge-Based Systems 145 (2018), 91–97.

64.

Shuai Zheng , Junge Zhang , Kaiqi Huang , Ran He , Tieniu Tan Robust viewtransformation model for gait recognition. In Image Processing (ICIP), 2011 18th IEEE International Conference on, pages 2073–2076. IEEE, 2011. http://www.cbsr.ia.ac.cn/users/szheng/?page id=71

65.

Zhaohui Zheng , Xiaoyun Wu , Rohini Srihari , Feature selection fortext categorization on imbalanced data, ACM Sigkdd ExplorationsNewsletter 6(1) (2004), 80–89.

66.

Ya Zhou , Atreyi Kankanhalli AI Regulation for Smart Cities: Challenges and Principles, pages 101–118. Springer International Publishing, Cham, 2021. ISBN 978-3-030-61033-3. doi: 10.1007/978-3-030-61033-3 5. URL https: //doi.org/10.1007/978-3-030-61033-3 5